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Chapter 1 

Introduction 


This Chapter begins with a high-level definition of the concepts of signals and systems. This is followed 
by a general introduction to digital signal processing (DSP), including an overview of the basic components 
of a DSP system and their functionality. Some practical examples of real-life DSP applications are then 
discussed briefly. The Chapter ends with a presentation of the course outline. 

1.1 The concepts of signal and system 

Signal: 

• A signal can be broadly defined as any quantity that varies as a function of time and/or space and has 
the ability to convey information. 

• Signals are ubiquitous in science and engineering. Examples include: 

- Electrical signals: currents and voltages in AC circuits, radio communications signals, audio and 
video signals. 

- Mechanical signals: sound or pressure waves, vibrations in a structure, earthquakes. 

- Biomedical signals: electro-encephalogram, lung and heart monitoring, X-ray and other types 
of images. 

- Finance: time variations of a stock value or a market index. 

• By extension, any series of measurements of a physical quantity can be considered a signal (tempera¬ 
ture measurements for instance). 


Signal characterization: 

• The most convenient mathematical representation of a signal is via the concept of a function, say x(t). 
In this notation: 

- x represents the dependent variable (e.g., voltage, pressure, etc.) 

- t the represents the independent variable (e.g., time, space, etc.). 
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• Depending on the nature of the independent and dependent variables, different types of signals can be 
identified: 

- Analog signal: t G M —> x a (t) SlorC 

When t denotes the time, we also refer to such a signal as a continuous-time signal. 

- Discrete signal: n£ Z —> x[n] slorC 

When index n represents sequential values of time, we refer to such a signal as discrete-time. 

- Digital signal: «£Z-> x[n\ S A 

where A = {a \,..., ny,} represents a finite set of L signal levels. 

- Multi-channel signal: x(t) = (x\(t),... ,xw(t )) 

- Multi-dimensional signal: x(t\ 

• Distinctions can also be made at the model level, for example: whether x[n] is considered to be 
deterministic or random in nature. 


Example 1.1: Speech signal 

► A speech signal consists of variations in air pressure as a function of time, so that it basically represents 
a continuous-time signal x(t). It can be recorded via a microphone that translates the local pressure 
variations into a voltage signal. An example of such a signal is given in Figure 1.1(a), which repre¬ 
sent the utterance of the vowel “a”. If one wants to process this signal with a computer, it needs to be 
discretized in time in order to accommodate the discrete-time processing capabilities of the computer 
(Figure 1.1(b)), and also quantized, in order to accommodate the finite-precision representation in a 
computer (Figure 1.1(b)). These represent a continuous-time, discrete-time and digital signal respec¬ 
tively. 

As we know from the sampling theorem, the continuous-time signal can be reconstructed from its sam¬ 
ples taken with a sampling rate at least twice the highest frequency component in the signal. Speech 
signals exhibit energy up to say, 10 kHz. However, most of the intelligibility is conveyed in a band¬ 
width less than 4 kHz. In digital telephony, speech signals are filtered (with an anti-aliasing filter which 
removes energy above 4 kHz), sampled at 8 kHz and represented with 256 discrete (non-uniformly- 
spaced) levels. Wideband speech (often termed commentary quality) would entail sampling at a higher 
rate, often 16 kHz. ◄ 


Example 1.2: Digital Image 

► An example of two-dimensional signal is a grayscale image, where t \ and ti_ represent the horizontal 
and vertical coordinates, and x(t\,t 2 ) represents some measure of the intensity of the image at location 
(fijfr)- This example can also be considered in discrete-time (or rather in discrete space in this case): 
digital images are made up of a discrete number of points (or pixels ), and the intensity of a pixel can be 
denoted by x[n i ,« 2 ]- Figure 1.2 shows an example of digital image. The rightmost part of the Figure 
shows a zoom on this image that clearly shows the pixels. This is an 8-bit grayscale image, i.e. each 
pixel (each signal sample) is represented by an 8-bit number ranging from 0 (black) to 255 (white). ◄ 


System: 

• A physical entity that operates on a set of primary signals (the inputs) to produce a corresponding set 
of resultant signals (the outputs). 


© B. Champagne & F. Labeau 


Compiled January 23, 2004 




1.1 The concepts of signal and system 


3 


(a) 



Fig. 1.1 An utterance of the vowel “a” in analog, discrete-time and digital format. Sampling 
at 4 kHz and quantization on 4 bits (16 levels). 
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Chapter 1. Introduction 



Fig. 1.2 Digital image, and zoom on a region of the image to show the pixels 

• The operations, or processing, may take several forms: modification, combination, decomposition, 
filtering, extraction of parameters, etc. 


System characterization: 

• A system can be represented mathematically as a transformation between two signal sets, as in x[n\ E 
Si — >y[n] = T{x[n ]} E &. This is illustrated in Figure 1.3 


x[n] 


T{.} 


y[n] 


Fig. 1.3 A generic system 


• Depending on the nature of the signals on which the system operates, different basic types of systems 
may be identified: 

- Analog or continuous-time system: the input and output signals are analog in nature. 

- Discrete-time system: the input and output signals are discrete. 

- Digital system: the input and outputs are digital. 

- Mixed system: a system in which different types of signals (i.e. analog, discrete and/or digital) 
coexist. 
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1.2 Digital signal processing 

1.2.1 Digital processing of analog signals 
Discussion: 

• Early education in engineering focuses on the use of calculus to analyze various systems and processes 
at the analog level: 

- motivated by the prevalence of the analog signal model 

- e.g.: circuit analysis using differential equations 

• Yet, due to extraordinary advances made in micro-electronics, the most common/powerful processing 
devices today are digital in nature. 

• Thus, there is a strong, practical motivation to cany out the processing of analog real-world signals 
using such digital devices. 

• This has lead to the development of an engineering discipline know as digital signal processing (DSP). 


Digital signal processing (DSP): 

• In its most general form, DSP refers to the processing of analog signals by means of discrete-time 
operations implemented on digital hardware. 

• From a system viewpoint, DSP is concerned with mixed systems: 

- the input and output signals are analog 

- the processing is done on the equivalent digital signals. 


1.2.2 Basic components of a DSP system 
Generic structure: 

• In its most general form, a DSP system will consist of three main components, as illustrated in Fig¬ 
ure 1.4. 

• The analog-to-digital (A/D) converter transforms the analog signal xd{t) at the system input into a 
digital signal Xd [n]. An A/D converter can be thought of as consisting of a sampler (creating a discrete¬ 
time signal), followed by a quantizer (creating discrete levels). 

• The digital system performs the desired operations on the digital signal y/ [n] and produces a corre¬ 
sponding output yd [n] also in digital form. 

• The digital-to-analog (D/A) converter transforms the digital output yj\n] into an analog signal y a (t) 
suitable for interfacing with the outside world. 

• In some applications, the A/D or D/A converters may not be required; we extend the meaning of DSP 
systems to include such cases. 
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Fig. 1.4 A digital signal processing system 


A/D converter: 

• A/D conversion can be viewed as a two-step process: 


(0 





Sampler 


x[n\ 


> 


Quantizer 


x d [n] 


Fig. 1.5 A/D conversion 


• Sampler: in which the analog input is transformed into a discrete-time signal, as in %,(t) —> x[n] = 
x a (nT s ), where T s is the sampling period. 

• Quantizer: in which the discrete-time signal x[n] £ M is approximated by a digital signal 47 \n] £ A, 
with only a finite set A of possible levels. 

• The number of representation levels in the set A is hardware defined, typically $ where b is the 
number of bits in a word. 

• In many systems, the set of discrete levels is uniformly spaced. This is the case for instance for WAVE 
files used to store audio signals. For WAVE files, the sampling rate is often 44.1 kHz (the sampling 
rate used for audio CD’s) or 48 kHz (used in studio recording). The level resolution for WAVE files 
is most often 16 bits per sample, but some systems use up to 24 bits per samples. 

• In some systems, the set of discrete levels is non-uniformly spaced. This is the case for digital tele¬ 
phony. Nearly all telephone calls are digitized to 8 bit resolution. However, the levels are not equally 
spaced — the spacing between levels increases with increasing amplitude. 
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Digital system: 

• The digital system is functionally similar to a microprocessor: it has the ability to perform mathemat¬ 
ical operations on a discrete-time basis and can store intermediate results of computation in internal 
memory. 

• The operations performed by the digital system can usually be described by means of an algorithm, 
on which its implementation is based. 

• The implementation of the digital system can take different forms: 

- Hardwired: in which dedicated digital hardware components are specially configured to accom¬ 
plish the desired processing task. 

- Softwired: in which the desired operations are executed via a programmable digital signal pro¬ 
cessor (PDSP) or a general computer programmed to this end. 

• The following distinctions are also important: 

- Real-time system: the computing associated to each sampling interval can be accomplished in a 
time < the sampling interval. 

- Off-line system: A non real-time system which operates on stored digital signals. This requires 
the use of external data storage units. 


D/A converter: 

This operation can also be viewed as a two-step process, as illustrated in Figure 1.6. 


y d M 


Pulse train 
gererator 


y a (t) 


Interpolator 


y a ( 0 


Fig. 1.6 D/A Conversion 


• Pulse train generator: in which the digital signal y r j [n] is transformed into a sequence of scaled, analog 
pulses. 

• Interpolator: in which the high frequency components of y a (t) are removed via low-pass filtering to 
produce a smooth analog output y a (t). 

This two-step representation is a convenient mathematical model of the actual D/A conversion, though, in 
practice, one device takes care of both steps. 
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1.2.3 Pros and cons of DSP 
Advantages: 

• Robustness: 

- Signal levels can be regenerated. For binary signals, the zeros and ones can be easily distin¬ 
guished even in the presence of noise as long as the noise is small enough. The process of 
regeneration make a hard decision between a zero and a one, effectively stripping off the noise. 

- Precision not affected by external factors. This means that one gets the results are reproducible. 

• Storage capability: 

- DSP system can be interfaced to low-cost devices for storage. The retrieving stored digital 
signals (often in binary form) results in the regeneration of clean signals. 

- allows for off-line computations 

• Flexibility: 

- Easy control of system accuracy via changes in sampling rate and number of representation bits. 

- Software programmable =>■ implementation and fast modification of complex processing func¬ 
tions (e.g. self-tunable digital filter) 

• Structure: 

- Easy interconnection of DSP blocks (no loading problem) 

- Possibility of sharing a processor between several tasks 


Disadvantages: 

• Cost/complexity added by A/D and D/A conversion. 

• Input signal bandwidth is technology limited. 

• Quantization effects. Discretization of the levels adds quantization noise to the signal. 

• Simple conversion of a continuous-time signal to a binary stream of data involves an increase in the 
bandwidth required for transmission of the data. This however can be mitigated by using compression 
techniques. For instance, coding an audio signal using MP3 techniques results in a signal which uses 
much less bandwidth for transmission than a WAVE file. 


1.2.4 Discrete-time signal processing (DTSP) 

Equivalence of analog and digital signal processing 

• It is not at all clear that an arbitrary analog system can be realized as a DSP system. 

• Fortunately, for the important class of linear time-invariant systems, this equivalence can be proved 
under the following conditions: 
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- the number of representation levels provided by the digital hardware is sufficiently large that 
quantization errors may be neglected. 

- the sampling rate is larger than twice the largest frequency contained in the analog input signal 
(Sampling Theorem). 

The DTSP paradigm 

• Based on these considerations, it is convenient to break down the study of DSP into two distinct sets 
of issues: 

- Discrete-time signal processing (DTSP) 

- Study of quantization effects 

• The main object of DTSP is the study of DSP systems under the assumption that finite-precision 
effects may be neglected =>■ DTSP paradigm. 

• Quantization is concerned with practical issues resulting from the use of finite-precision digital hard¬ 
ware in the implementation of DTSP systems. 

1.3 Applications of DSP 

Typical applications: 

• Signal enhancement via frequency selective filtering 

• Echo cancellation in telephony: 

- Electric echoes resulting from impedance mismatch and imperfect hybrids. 

- Acoustic echoes due to coupling between loudspeaker and microphone. 

• Compression and coding of speech, audio, image, and video signals: 

- Low bit-rate codecs (coder/decoder) for digital speech transmission. 

- Digital music: CD, DAT, DCC, MD,...and now MP3 

- Image and video compression algorithms such as JPEG and MPEG 

• Digital simulation of physical processes: 

- Sound wave propagation in a room 

- Baseband simulation of radio signal transmission in mobile communications 

• Image processing: 

- Edge and shape detection 

- Image enhancement 
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Example 1.3: Electric echo cancellation 

► In classic telephony (see Figure 1.7), the signal transmitted through a line must pass through a hybrid 
before being sent to the receiving telephone. The hybrid is used to connect the local loops deserving the 
customers (2-wire connections) to the main network (4-wire connections). The role of the hybrid is to 
filter incoming signals so that they are oriented on the right line: signals coming from the network are 
passed on to the telephone set, signals from the telephone set are passed on to the transmitting line of 
the network. This separation is never perfect due to impedance mismatch and imperfect hybrids. For 
instance, the signal received from the network (on the left side of Figure 1.7) can partially leak into 
the transmitting path of the network, and be sent back to the transmitter (on the right side), with an 
attenuation and a propagation delay, causing an echo to be heard by the transmitter. 

To combat such electric echoes, a signal processing device is inserted at each end of the channel. The 
incoming signal is monitored, processed through a system which imitates the effect of coupling, and 
then subtracted from the outgoing signal. ◄ 


direct signal 


Receiver 


I 



1 


Transmitter 


Hybrid 


Hybrid 



echo signal 


Fig. 1.7 Illustration of electrical echo in classic telephone lines. 


Example 1.4: Edge detection 

► An edge detection system can be easily devised for grayscale images. It is convenient to study the 
principle in one dimension before going to two dimensions. Figure 1.8(a) illustrates what we expect an 
ideal edge to look like: it is a transition between two flat regions, two regions with approximately the 
same grayscale levels. Of course, in practise, non ideal edges will exist, with smoother transitions and 
non flat regions. For the sake of explanation, let us concentrate on this ideal edge. Figure 1.8(b) shows 
the impulse response of a filter that will enable edge detection: the sum of all its samples is equal to one, 
so that the convolution of a flat region with this impulse response will yield 0, as the central peak will be 
compensated by the equal side values. On the other hand, when the convolution takes place on the edge, 
the values on the left of the peak and on the right of the peak will not be the same anymore, so that he 
output will be nonzero. So to detect edges in a signal in an automatic way, one only has to filter it with 
a filter like the one shown in Figure 1.8(b) and threshold the output. A result of this procedure is shown 
on Figure 1.9 for the image shown earlier. ◄ 
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(a) (b) 

Fig. 1.8 (a) An ideal edge signal in one dimension and (b) an edge detecting filter 



Fig. 1.9 Illustration of an edge detection system on image of Figure 1.2. 
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1.4 Course organization 

Three main parts: 

• Part I: Basic tools for the analysis of discrete-time signals and systems 

- Time-domain and transform-domain analysis 

- Discrete-time Fourier transform (DTFT) and Z-transform 

- Discrete Fourier transform (DFT) 

• Part II: Issues related to the design and implementation of DSP systems: 

- Sampling and reconstruction of analog signals, A/D and D/A 

- Structures for realization of digital filters, signal flow-graphs 

- Digital filter design (FIR and HR) 

- Study of finite precision effects (quantization noise,...) 

- Fast computation of the DFT (FFT) 

- Programmable digital signal processors (PDSP) 

• Part III: Selected applications and advanced topics. 

- Applications of DFT to frequency analysis 

- Multi-rate digital signal processing 

- Introduction to adaptive filtering 
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Chapter 2 


Discrete-time signals and systems 


2.1 Discrete-time (DT) signals 

Definition: 

A DT signal is a sequence of real or complex numbers, that is, a mapping from the set of integers Z into 
either M or C, as in: 


n E Z —► x[n] 6 1 or C 


( 2 . 1 ) 


• n is called the discrete-time index. 

• x[n], the nth number in the sequence, is called a sample. 

• To refer to the complete sequence, one of the following notations may be used: x, \x[n ]} or even x[n] 
if there is no possible ambiguity. 1 

• Unless otherwise specified, it will be assumed that the DT signals of interest may take on complex 
values, i.e. x{n] E C. 

• We shall denote by S the set of all complex valued DT signals. 


Description: 


There are several alternative ways of describing the sample values of a DT signal. Some of the most common 
are: 


• Sequence notation: 



( 2 . 2 ) 


where the bar on top of symbol 1 indicates origin of time (i.e. n = 0) 


'The latter is a misuse of notation, since x[n\ formally refers to the nth signal sample. Following a common practice in the DSP 
literature, we shall often use the notation x[n\ to refer to the complete sequence, in which case the index n should be viewed as a 
dummy place holder. 
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• Graphical: 


x[n] 


1 


-•-•-•-•-•- 


L-i. 


-*-f- 


0 1 2 


Explicit mathematical expression: 


x\n\ = 


0 n < 0, 
2~ n n > 0. 


Recursive approach: 


x[n ] = < 


n < 0, 
n = 0, 


k \x[n — 1] n > 0 


(2.3) 


(2.4) 


Depending on the specific sequence x[n], some approaches may lead to more compact representation than 
others. 


Some basic DT signals: 


• Unit pulse: 


SM 


1 n = 0, 

0 otherwise. 


• Unit step: 


u[n] 


1 n > 0, 
0 n < 0. 


• Exponential sequence: 
for some a e C 


x[n\=Aa n , «GZ 


(2.5) 


S[n] 


0 1 2 



(2.7) 


1 




-f-► 

n 
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• Complex exponential sequence (CES): 

If |a| = 1 in (2.6), i.e. a = e • /a) for some co £ K, we have 

x[n]=Ae j(0n , n£Z (2.8) 

where to is called the angular frequency (in radian). 

- One can show that a CES is periodic with period N, i.e. x[n+N] = jt[n], if and only if to = 2nk/N 
for some integer k. 

- If a >2 = co i +27t, then the two CES signals X 2 [n] = e /a ’ 2 " and x\ [n] = e ,0)| " are indistinguishable 
(see also Figure 2.1 below). 

- Thus, for DT signals, the concept of frequency response is really limited to an interval of size 
27t, typically [—7C,7t]. 


Example 2.1: Uniform sampling 

► In DSP applications, a common way of generating DT signals is via uniform (or periodic) sampling of 
an analog signal x a (t),t £l, as in: 

x[n] = x a (nT s ) , n£ Z (2.9) 

where 7^ > 0 is called the sampling period. 

For example, consider an analog complex exponential signal given by x a (t) = e jlKFl , where F denotes 
the analog frequency (in units of 1/time). Uniform sampling of x a (t) results in the discrete-time CES 

x[n]=e j2aFnT ‘ = e jm (2.10) 

where co = 2nFT s is the angular frequency (dimensionless). 

Figure 2.1 illustrates the limitation of the concept of frequencies in the discrete-time domain. The 
continuous and dashed-dotted lines respectively show the real part of the analog complex exponential 
signals e' w and e J<m 1 2 ' x ''. Upon uniform sampling at integer values of t (i.e. using T s = 1 in (2.9)), the 
same sample values are obtained for both analog exponentials, as shown by the solid bullets in the figure. 

That is, the DT CES e J<,> " and e J<>> 2n> " are indistinguishable, even tough the original analog signals are 
different. This is a simplified illustration of an important phenomenon known as frequency aliasing. ◄ 

As we saw in the example, for sampled data signals (discrete-time signals formed by sampling continuous¬ 
time signals), there are several frequency variables: F the frequency variable (units Hz) for the frequency 
response of the continuous-time signal, and co the (normalized) radian frequency of the frequency response 
of the discrete-time signal. These are related by 

co = 27 iF/F s , (2.11) 

where F s (F s = 1 /T s ) is the sampling frequency. One could, of course, add a radian version of F (Q. = 2nF) 
or a natural frequency version of co (f = co/(27t)). 

For a signal sampled at F s , the frequency interval —F s /2 <F< F s /2 is mapped to the interval — 7t < co < 7t. 


Basic operations on signal: 

• Let S denote the set of all DT signals. 
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Fig. 2.1 Illustration of the limitation of frequencies in discrete-time. 


• We define the following operations on S : 

scaling: (ax)[n\ = ax[n\, where a £ C (2.12) 

addition: {x+y)[n]= x[n}+y[n] (2.13) 

multiplication: (xy) [n] = jt[n]y[n] (2.14) 

• Set S equipped with addition and scaling is a vector space. 

Classes of signals: 

The following subspaces of S play an important role: 

• Energy signals: all x £ S with finite energy, i.e. 

oo 

£,= £ |x[«]| 2 <oo (2.15) 

n——oo 

• Power signals: all x £ S with finite power, i.e. 

1 N 

<P X = lim - y Ixl/tl | 2 < °° (2.16) 

2N + 1 n "~!_ N 

• Bounded signals: all x E S that can be bounded, i.e. we can find B x > 0 such that |x[/i] < B x for all 

n£Z 

• Absolutely summable: all x E S such that Ln=- 00 I*Ml <°° 

Discrete convolution: 

• The discrete convolution of two signals x and y in S is defined as 

00 

(x*y)[n\= Y, x[k]y[n-k] (2.17) 

k ——00 

The notation x[n] *y[n\ is often used instead of (x*y)[n\. 
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• The convolution of two arbitrary signals may not exist. That is, the sum in (2.17) may diverge. 
However, if both ,r and y are absolutely summable, x *y is guaranteed to exist. 

• The following properties of * may be proved easily: 


(a) 

x*y = y*x 

(2.18) 

(b) 

(x*y)*z = x*(y*z) 

(2.19) 

(c) 

x*8 = x 

( 2 . 20 ) 


• For example, (a) is equivalent to 

oo oo 

x[k]y[n — k]= ^ y[k]x[n — k] (2.21) 

k——oo k ——oo 

which can be proved by changing the index of summation from k to k' = n — k in the LHS summation 
(try it!) 


2.2 DT systems 


Definition: 


A DT system is a mapping T from S into itself. That is 

x£S—>y = Tx£S ( 2 . 22 ) 

An equivalent block diagram form is shown in Figure 2.2 We refer to x as the input signal or excitation, and 
to y as the output signal, or response. 


x[n],n G Z 


input 


T{.} 


y[n],neZ 

-► 

output 


Fig. 2.2 A generic Discrete-Time System seen as a signal mapping 


• Note that y[n], the system output at discrete-time n, generally depends on x[k] for all values of k G Z. 

• Even tough the notation y[n] = T {x\n} } is often used, the alternative notation y[n] = (Tx) [n] is more 
precise. 

• Some basic systems are described below. 

Time reversal: 

• Definition: 

y[n) = ( Rx)[n\ — x[—n) (2.23) 

• Graphical interpretation: mirror image about origin (see Figure 2.3) 
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Fig. 2.3 Illustration of time reversal: left, original signal; right: result of time reversal. 


Delay or shift by integer k: 

• Definition: 

y[n\ = (Dkx)[n\ = x[n — k) (2.24) 

• Interpretation: 

- k>0=> graph of x[n] shifted by k units to the right 

- k < 0 =>• graph of x[n] shifted by \k\ units to the left 

• Application: any signal x E S can be expressed as a linear combination of shifted impulses: 

oo 

x[n]= ^ x[k\8[n — k] (2.25) 


Other system examples: 

• Moving average system: 

1 v 

y\n] = - V x\n — k] (2.26) 

M + N+ \ k t M 

This system can be used to smooth out rapid variations in a signal, in order to easily view long-term 
trends. 

• Accumulator: 

y[n\ = y x [k\ (2.27) 

k=—oo 


Example 2.2: Application of Moving Average 

► An example of Moving Average filtering is given in Figure 2.4: the top part of the figure shows daily 
NASDAQ index values over a period of almost two years. The bottom part of the figure shows the same 
signal after applying a moving average system to it, with M + N + 1 = 10. Clearly, the output of the 
moving average system is smoother and enables a better view of longer-term trends. ◄ 
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Fig. 2.4 Effect of a moving average filter. (Sample values are connected by straight lines to 
enable easier viewing) 


Basic system properties: 

• Memory less: y[n] = (Tx) [n] is a function of x[n\ only. 

Example 2.3: 

► y[n] = (x[n\) 2 is memoryless, but y\n] = \(x[n — 1] + x[n]) is not. ◄ 

Memoryless systems are also called static systems. Systems with memory are termed dynamic sys¬ 
tems. Further, systems with memory can have finite (length) memory or infinite (length) memory. 

• Causal: y[n] only depends on values x[k] for k < n. 

• Anti-causal: y[n] only depends on values x[k] for k > n 2 . 

Example 2.4: 

► y[n] = (Tx)[n] = i s causal, y[n] = YX=n x \^\' s anti-causal. ◄ 

• Linear: for any ai, ai £ C and x\,xj € S, 

T(a\x\ + CL 2 X 2 ) = a\T(x{) -\-d 2 T(x 2 ) (2.28) 

2 This definition is not consistent in the literature. Some authors prefer to define an anti-causal system as one where y[«] only 
depends on values x[k\ for k> n, i.e. the present output does only depend on future inputs, not even the present input sample. 
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Example 2.5: 

► The moving average system is a good example of linear system, but e.g. y [n] = (x[n]) 2 is obviously 
not linear. ◄ 

• Time-invariant: for any iGZ, 

T(D k x)=D k {Tx) (2.29) 

or equivalently, T{x[n ]} = y[n) =>■ T{x[n — k ]} = y[n — k\. 


Example 2.6: 

► The moving average system can easily be shown to be time invariant by just replacing n by n — k 
in the system definition (2.26). On the other hand, a system defined by y[n] = (Tx) [n] = x[2n], i.e. 
a decimation system, is not time invariant. This is easily seen by considering a delay of k = 1: 
without delay, y[n] = x\2n}\ with a delay of 1 sample in the input, the first sample of the output is 
x[— 1], which is different from y[— 1] = x[— 2]. ◄ 

• Stable: x bounded => y = Tx bounded. That is, if \x\n] | < B x for all n, then we can find B y such that 
y[/i] j < By for all n 


Remarks: 

In on-line processing, causality of the system is very important. A causal system only needs the past and 
present values of the input to compute the current output sample y[n], A non-causal system would require 
at least some future values of the input to compute its output. If only a few future samples are needed, this 
translates into a processing delay, which may not be acceptable for real-time processing; on the other hand, 
some non-causal systems require the knowledge of all future samples of the input to compute the current 
output sample: these are of course not suited for on-line processing. 

Stability is also a very desirable property of a system. When a system is implemented, it is supposed to meet 
certain design specifications, and play a certain role in processing its input. An unstable system could be 
driven to an unbounded output by a bounded input, which is of course an undesirable behaviour. 


2.3 Linear time-invariant (LTI) systems 

Motivation: 

Discrete-time systems that are both linear and time-invariant (LTI) play a central role in digital signal pro¬ 
cessing: 

• Many physical systems are either LTI or approximately so. 

• Many efficient tools are available for the analysis and design of LTI systems (e.g. Fourier analysis). 
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Fundamental property: 

Suppose T is LTI and let y = Tx(x arbitrary). Then 


y[n] = ^ x[k\h{n — k\ 

k——oo 


where h = T8 is known as the unit pulse (or impulse) response of T. 
Proof: Recall that for any DT signal x, we can write 

oo 

x = Y x[k\ D k 8 

k——oo 

Invoking linearity and then time-invariance of T, we have 


oo 

y = Tx= Y x[k\T(D k 8) 

k =—oo 

oo oo 

= Y x[k\D k (TS)= Y x[k]D k h 

k =—oo k ——oo 


which is equivalent to (2.30) □. 


(2.30) 


(2.31) 


(2.32) 

(2.33) 


Discussion: 

• The input-output relation for the LTI system may be expressed as a convolution sum: 

y =x*h (2.34) 

• For LTI system, knowledge of impulse response h = T$ completely characterizes the system. For any 
other input x, we have Tx = x*h. 

• Graphical interpretation: to compute the sample values of y[n] according to (2.34), or equivalently (2.30), 
one may proceed as follows: 

- Time reverse sequence h[k] => h[—k] 

- Shift h[—k\ by n samples => h[—(k — n )] = h[n — k] 

- Multiply sequences x[k] and /; [/; — k] and sum over k => y[n\ 

Example 2.7: Impulse response of the accumulator 

► Consider the accumulator given by (2.27). There are different ways of deriving the impulse response 
here. By setting x[n] = 8[n] in (2.27), we obtain: 

h[n)=Y 8 W = jo 

= u[n\ 


n > 0, 
n < 0. 


(2.35) 
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An alternative approach is via modification of (2.27), so as to directly reveal the convolution format 
in (2.30). For instance, by simply changing the index of summation from k to / = n — k , equation (2.27) 
then becomes: 


y[n) = £*[«-*] 


1=0 


= Y — /], 

/=-so 


so that by identification, h[n) = u[n ]. 


◄ 


Causality: 

An LTI system is causal iff h\n\ = 0 for n < 0. 

Proof: The input-output relation for an LTI system can be expressed as: 

oo 

y[n] = Y h[k]x[n — k\ (2.36) 

k——oo 

= - \-h[—\]x[n + 1] + /j[0]x[«] + /z[l]x[n — 1] H- (2.37) 

Clearly, y[n] only depends on values x[m] for m < n iff h[k] = 0 for k < 0 □. 


Stability: 

An LTI system is stable iff the sequence h[n\ is absolutely summable, that is \h[n]\ < oo . 

Proof: Suppose that h is absolutely summable, that is £„ \h[n\ \ = M] x < oo. Then, for any bounded input 
sequence x, i.e. such that | x[n] \ < B x < °o, we have for the corresponding output sequence y: 

bM I = I Y x \. n - kl \ h WI - L \ x i n - k 1 1 bW I 

k k 

<Y B *\h[k\\ = B x M h < oo (2.38) 

k 

which shows that y is bounded. Now, suppose that h\n\ is not absolutely summable, i.e. \h\n}\ = °o. 

Consider the input sequence defined by 


x[—n\ 


h [n] * /1 h [n] \ if li [n] fo 0 
0 if h[n] = 0. 


Note that |jc[/j]| < 1 so that x is bounded. We leave it as an exercise to the student to verify that in this case, 
y[0] = +oo, so that the output sequence y is unbounded and so, the corresponding LTI system is not bounded. 

□ 
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Example 2.8: 

► Consider LTI system with impulse response h[n] = a n u\n\ 

• Causality: h[n] = 0 for n < 0 => causal 

• Stability: 

t \h[n]\=£\a\ n 

n=~oo n=0 

Clearly, the sum diverges if |a| > 1, while if |a| < 1, it converges: 


E l a l 

77=0 


Thus the system is stable provided |a| < 1. 


1 

l-|a| 


oo 


(2.39) 


(2.40) 


◄ 


FIR versus HR 


An LTI system has finite impulse response (FIR) if we can find integers N i < AL such that 

h[n] = 0 when n < N\ or n > N 2 

Otherwise the LTI system has an infinite impulse response (HR). 

For example, the LTI system with 

. . . . . . f 1 0<7t<A- 1 

h[n\ = u[n\-u[n-N\ = < . 

I 0 otherwise 

is FIR with N[ = 0 and N 2 = N — 1. The LTI system with 

a" n > 0 


h[n] = a ll u[n] = 


0 otherwise 


is HR (cannot find an N 2 ...) 

FIR systems are necessarily stable: 


n 2 


E IM«]I = E l /7 HI < 

i=—°° n=N\ 


(2.41) 


(2.42) 


(2.43) 


(2.44) 


Interconnection of LTI systems: 

• Cascade interconnection (Figure 2.5): 

y = [x*hi)* fi2 =x* (Ji 1 * I12 ) 
= x* {]l2*h\) = (x*Ii2) *h 1 
LTI systems commute; not true for arbitrary systems. 

• Parallel interconnection (Figure 2.6): 

y = x*h\+x*h.2=x* (hi +h 2 ) 


(2.45) 

(2.46) 


(2.47) 
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Fig. 2.5 Cascade interconnection of LTI systems 



Fig. 2.6 Parallel interconnection of LTI systems 


2.4 LTI systems described by linear constant coefficient difference equations (LCCDE) 

Definition: 

A discrete-time system can be described by an LCCDE of order N if, for any arbitrary input ^ and corre¬ 
sponding output y, 

N M 

52 a k y[n - k) = 52 h*[n - k ] (2.48) 

i =0 k= o 

where qq ^ 0 and cry / 0. 


Example 2.9: Difference Equation for Accumulator 

► Consider the accumulator system: 

x[n) — >y[n)= 52 x [k] (2.49) 

k——oo 

which we know to be LTI. Observe that 

n— 1 

y[n]= 52 x[k]+x[n] 

k=-oo 

= y[n — 1] + x[n] (2.50) 

This is an LCCDE of order A = 1 (M = 0, «o = L a\ = —1, = 1) ◄ 
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Remarks: 

LCCDEs lead to efficient recursive implementation: 

• Recursive because the computation of y[n] makes use past output signal values (e.g. y[n — 1] in (2.50)). 

• These past output signal values contain all the necessary information about earlier states of the system. 

• While (2.49) requires an infinite number of adders and memory units, (2.50) requires only one adder 
and one memory unit. 

Solution of LCCDE: 

The problem of solving the LCCDE (2.48) for an output sequence y[n], given a particular input sequence 
x[n], is of particular interest in DSR Here, we only look at general properties of the solutions y[n\. The 
presentation of a systematic solution technique is deferred to Chapter 4. 

Structure of the general solution: 

The most general solution of the LCCDE (2.48) can be expressed in the form 

y[n\=y p [n\+y k [n] (2.51) 

• y p [n] is any particular solution of the LCCDE 

• yi, [n is the general solution of the homogeneous equation 

N 

52 a k y h [n-k}= 0 (2.52) 

k =o 

Example 2.10: 

► Consider the 1st order LCCDE 

y[n] = ay[n — 1] -\-x[n] (2.53) 

with input x[n] = A8[;?]. 

It is easy to verify that the following is a solution to (2.53): 

y p \n] = Aa" u\n\ (2.54) 

The homogeneous equation is 

yh[n\+ay h [n- 1] = 0 (2.55) 

Its general solution is given by 

y h [n]=Ba n (2.56) 

where B is an arbitrary constant. So, the general solution of the LCCDE is given by: 

y[n]=Aa n u[n\+Ba n (2.57) 

◄ 
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Remarks on example: 

• The solution of (2.53) is not unique: B is arbitrary 

• (2.53) does not necessarily define a linear system: in the case A = 0 and B = 1, we obtain y[n) / 0 
while x[n] = 0. 

• The solution of (2.53) is not causal in general: the choice B = —A gives 



(2.58) 


which is an anti-causal solution. 

General remarks: 

• The solution of an LCCDE is not unique; for an Mh order LCCDE, uniqueness requires the specifi¬ 
cation of N initial conditions. 

• An LCCDE does not necessarily correspond to a causal LTI system. 

• However, it can be shown that the LCCDE will correspond to a unique causal LTI system if we further 
assume that this system is initially at rest. That is: 


x[n\ = 0 for n < no 


y[n] = 0 for n < no 


(2.59) 


• This is equivalent to assuming zero initial conditions when solving LCCDE, that is: y[n o — /] = 0 for 

/=!,...,A. 


• Back to example: Here x[n\ = A'8[n] = 0 for n < 0. Assuming zero-initial conditions, we must have 
y[— 1] = 0. Using this condition in (2.57), we have 


y[—1] =Ba~ l = Q^B = 0 

so that finally, we obtain a unique (causal) solution y\n\ =Aa"u\n]. 

• A systematic approach for solving LCCDE will be presented in Chap. 4. 


(2.60) 


2.5 Problems 


Problem 2.1: Basic Signal Transformations 

Let the sequenc x[n] be as illustrated in figure 2.7. Determine and draw the following sequences: 


1. x[—n\ 


2. x[n — 2] 

3. x[3 — n] 


© B. Champagne & F. Labeau 


Compiled September 13, 2004 




2.5 Problems 


27 


4. x[n] * 8 [n — 3] 

5. x[n\ * (u[n\ — u[n — 2]) 

6. x[2n]. 



Fig. 2.7 Sequence to be used in problem 2.1. 


Problem 2.2: Impulse response from LCCDE 

Let the input-output relationship of an LTI system be described by the following difference equation: 

y[n] + -y[n - 1 ] = x[n\ - 4 x[n - 1 ], 

together with initial rest conditions. 

1. Determine the impulse response h[n] of the corresponding system. 

2. Is the system causal, stable, FIR, HR ? 

3. Determine the output of this system if the input x[n\ is given by x[n] = j ... , 0,0, -, 0, — 1,0,0,0,... 


Problem 2.3: 

Let Tj,i = 1,2,3, be stable LTI systems, with corresponding impulse responses hi[n]. 

1. Is the system illustrated in figure 2.8 LTI ? If so, prove it, otherwise give a counter-example. 

2. If the system is LTI, what is its impulse response ? 


Problem 2.4: 

Let Ti, i = 1,2, be LTI systems, with corresponding impulse responses 
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Fig. 2.8 System connections to be used in problem 2.3. 


T{.} 



Fig. 2.9 Feedback connection for problem 2.4. 
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1. Write a linear constant coefficient difference equation (LCCDE) corresponding to the input-output 
relationships of T\ and Ti. 

2. Consider the overall system T in figure 2.9, with impulse response h[n\. Write a LCCDE correspond¬ 
ing to its input-output relationship. Explain how you could compute h[n] from this LCCDE. Compute 
h[0], h[\] and h[2}. 
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Chapter 3 

Discrete-time Fourier transform (DTFT) 


3.1 The DTFT and its inverse 


Definition: 

The DTFT is a transformation that maps DT signal x[n] into a complex-valued function of the real variable, 
namely 


oo 

X(co) = £ x[n]e- j(0n , co e R 

n ——oo 


(3.1) 


Remarks: 

• In general, X(oo) E C 

• X(oo + 27t) = X(co) => co G [—Ji, Jt] is sufficient 

• X(co) is called the spectrum of x[n]: 

x(to) = pfNi^w =► ( |x(m|1 = maEnitude spectrum ' 

I ZX(co) = phase spectrum. 

The magnitude spectrum is often expressed in decibel (dB): 

|X(co)| dB =201og 10 |X(co)| 


(3.2) 


(3.3) 


Inverse DTFT: 

Let X(co) be the DTFT of DT signal x[n). Then 


x[n] 


J- [ X(u>)e jan d(0, 

271 J —TI 


n G Z 


(3.4) 
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Proof: First note that f 7l 7l e /( " n do) = 2jt8[n]. Then, we have 


X(co)e J(0n d(0= / { £ x[k]e~ jak }e jan d(0 

' k =-oo 

00 rK 

= £ x[k] / e j(0 ^d(0 


k=- 


= 27t £ x[k] 8[n — k] = 2nx[n\ □ 

k——oo 


(3-5) 


Remarks: 

• In (3.4), x[n] is expressed as a weighted sum of complex exponential signals e ;co ", to e [—7t,7t], with 
weight X(oo) 

• Accordingly, the DTFT X((o) describes the frequency content of x[n\ 

• Since the DT signal x[n] can be recovered uniquely from its DTFT X(co), we say that x[n] together 
with X(a) form a DTFT pair, and write: 

x[n]<-?-^X( co) (3.6) 

• DTFT equation (3.1) is called analysis relation; while inverse DTFT equation (3.4) is called synthesis 
relation. 

3.2 Convergence of the DTFT: 

Introduction: 

• For the DTFT to exist, the series L“ = _oo x[n]e~j (0n must converge 

• That is, the partial sum 

M 

X M (oo)= £ x[n\e~j m (3.7) 

n=—M 

must converge to a limit X(co) as M —> °o. 

• Below, we discuss the convergence of Xm( co) for thicc different signal classes of practical interest, 
namely: 

- absolutely summable signals 

- energy signals 

- power signals 

• In each case, we state without proof the main theoretical results and illustrate the theory with corre¬ 
sponding examples of DTFTs. 
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Absolutely summable signals: 

• Recall: x[n] is said to be absolutely summable (sometimes denoted as L \) iff 

oo 

52 \x[n]\ <°° (3.8) 

n——oo 

• In this case, X(co) always exists because: 

oo oo oo 

| 52 x[n]e~ j(an \< 52 \x[n]e~ jan \= 52 \x[n\\<oo (3.9) 

fl —— 00 Tl —— 00 fl —— 00 

• Possible to show that Xm( to) converges uniformly toX(co), that is: 

For all 8 > 0, can find M e such that \X (to) — Xm{ to) | < 8 for all M > M e and for all to G M. 

• X(co) is continuous and d p X(a>)/d(O p exists and continuous for all p > 1. 


Example 3.1: 

► Consider the unit pulse sequence x[n] = 8[«]. The corresponding the DTFT is simply X(or) = 1. 

More generally, consider an arbitrary finite duration signal, say (N\ < N 2 ) 

n 2 

x\n\ = 52 Ck$[n — k] 
k=Ni 

Clearly, x[n] is absolutely summable; its DTFT is given by the finite sum 

n 2 

X(<Q) = 52 c n e-j°> n 

n=N\ 

◄ 


Example 3.2: 


► The exponential sequence x[n] 
puted to be 


a"u[n\ with \a\ < 1 is absolutely summable. Its DTFT is easily com- 


n —0 


1 

1 — ae~j a 


Figure 3.1 illustrates the uniform convergence of Aw(to) as defined in (3.7) to A(co) in the special case 
a = 0.8. As M increases, the whole An(to) curve tends to X(or) at every frequency. ◄ 
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Fig. 3.1 Illustration of uniform convergence for an exponential sequence. 


Energy signals: 


• Recall: x[n] is an energy signal (square summable or Ln) iff 

oo 

%c = Yj l x MI 2 < 00 

n——oo 


• In this case, it can be proved that Xm{ co) converges in the mean square sense to X(co), that is: 

/ it 

|X(oo) -X M {u>)\ 2 du> = 0 

-71 

• Mean-square convergence is weaker than uniform convergence: 

- L\ convergence implies £2 convergence. 

- £2 convergence does not guarantee that lim / v/^ 00 X^/(co) exists for all co. 

- e.g., X(co) may be discontinuous (jump) at certain points. 


Example 3.3: Ideal low-pass filter 

► Consider the DTFT defined by 


X(co) = 


1 |co| < C 0 c 
0 to,. < co < n 


whose graph is illustrated in Figure 3.2(a). The corresponding DT signal is obtained via (3.4) as follows: 


co, 


, or/1 


— / e J(0 "d co=—sinc( —) 

2jt J- co, n k 


(3.12) 


(3.10) 


(3.11) 
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(a) DTFT 


1 

0.8 


x 0.4 
0.2 
0 


-K -0) 0 CO 71 

c angular frequency to c 


(b) DT signal 



Fig. 3.2 Impulse response of the ideal low-pass filter. 

where the sine function is defined as 

sin7Df 

sincm =-. 

nx 

The signal x[n] is illustrated in Figure 3.2(b) for the case co c = Jt/2 
It can be shown that: 

• Y.n l x M | = 00 => x[n\ not absolutely summable 

• Ln |*[«] | 2 < 00 => x[n] square summable 

Here, Xm{ Co) converges to X (co) in the mean-square only; the convergence is not uniform. We have the 
so-called Gibb’s phenomenon (Figure 3.3): 

• There is an overshoot of size AX « 0.09 near ±C0 c ; 

• The overshoot cannot be eliminated by increasing M; 

• This is a manifestation of the non-uniform convergence; 

• It has important implications in filter design. 

◄ 


Power signals 

• Recall: x[n] is a power signal iff 




n——N 


(3.13) 
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M=5 


M=9 




M=15 M=35 




Fig. 3.3 Illustration of the Gibb’s phenomenon. 
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• If x[n\ has infinite energy but finite power, Xm(g>) may still converge to a generalized function X(co): 

• The expression of X (go) typically contains continuous delta functions in the variable to. 

• Most power signals do not have a DTFT even in this sense. The exceptions include the following 
useful DT signals: 

- Periodic signals 

- Unit step 

• While the formal mathematical treatment of generalized functions is beyond the scope of this course, 
we shall frequently make use of certain basic results, as developed in the examples below. 

Example 3.4: 

► Consider the following DTFT 


X(co) = 2ji Y 5 a (co-2jtk) 


where 8 a (co) denotes an analog delta function centred at CO = 0. By using the synthesis relation (3.4), 
one gets 



2jt Y e JCO "8„(co —27tk)dco 



(3.14) 



JOn _ i 


(3.15) 


since in (3.14) the only value of k for which the delta function will be non-zero in the integration interval 


is k = 0. 


Example 3.5: 


► Consider the unit step sequence u[n] and let U (co) denotes its DTFT. The following result is given without 
proof: 


U( co) = --Tto+rc E M®-2 nk) 


1 
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3.3 Properties of the DTFT 

Notations: 

• DTFT pairs: 

x[n ] X(co) 

y[n] y(eo) 


• Real and imaginary parts: 

x[n\ =x R [n] + jx I [n] 
X(co)=X i? (ro) + yX / (co) 


• Even and odd components: 

x[n] = x e [n\ + x 0 [n] (3.16) 

x e [n\ = -(jc[n] +x*[—n]) =x*[—n\ (even) (3.17) 

x 0 [n\ = -{x[n\ — x*[—n\) = —x* 0 [—n\ (odd) (3.18) 

• Similarly: 

X(oo) = X e (co)+X 0 (co) (3.19) 

X e ((i>) = i(X(co) + X*(-co)) =X*(-a) (even) (3.20) 

Xo(co) = ^(X(co) —X*(—to)) = —X*(—to) (odd) (3.21) 


Basic symmetries: 


x[—n] > 

X(-co) 

(3.22) 

* r i 

x l n \ < - > 

**(-«) 

(3.23) 

Xr[u] 

X e (co) 

(3.24) 

jxi[n] <-> 

X 0 (co) 

(3.25) 

x e [n] <-> 

X R (t b) 

(3.26) 

x 0 [n\ <-> 

JM co) 

(3.27) 
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Real/imaginary signals: 

• If x[n) is real, then X(co) = X*(— to) 

• If x[n\ is purely imaginary, thenX(oo) = — X*(— to) 

Proof: x[n] 6M4i;[n] =0=^X 0 (co) =0. In turns, this implies X (to) = X e (co) =X*(—to). Similar argument 
for purely imaginary case. □ 

Remarks: 

• For x[n] real, symmetry X (to) = X*(— co) implies 


|X(co)| = |X(-to)|, ZX(oo) =-ZX(-co) 
X R {(o)=X R (-(o), Xj( to) = —X/(—to) 


(3.28) 

(3.29) 


This means that, for real signals, one only needs to specify X(oo) for to £ [0,7c], because of symmetry. 


Linearity: 


<T 

ax[n] + by[n] <—> aX (co) + b Y (co) 


(3.30) 


Time shift (very important): 


x[n-d\^e~ jad X{ to) 


(3.31) 


Frequency modulation: 


e ja ° n x[n\ ^X((O-co 0 ) 


(3.32) 


Differentiation: 



(3.33) 


Example 3.6: 

► y[n) = box[n\ +b\x[n — 1]. By using the linearity and time shift properties, one gets that 

Y(a>) = (bo + b l e~ j(0 )X(a>) 
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Example 3.7: 

► y[n\ = cos(co 0 n)x[n]. Recall first that e- 7 ™ 0 " = cos(co 0 n) + j sin(co 0 «), so that cos(co 0 «) = + 

e“ ;ft) o"), and, by linearity and time shift: 

T(co) = i(X(co-co 0 )+X(co + co 0 )) 

◄ 


Convolution: 


x[n\ *y[n\ X(co)7(co) 


(3.34) 


Multiplication: 


x[n]y[n] — [ X((|))T((0 — 4>)<r/(() (3.35) 

2n J-n 

We refer to the RHS of (3.35) as the circular convolution of periodic functions X (oo) and Y (oo). 

Thus: 

- DT convolution *—> multiplication in to while 

<y 

- DT multiplication <—> circular convolution in co 
We say that these two properties are dual of each other 


Parseval’s relation: 

“ 1 f' n 

LkN| 2 = ^/ |*(co)| 2 r/« (3-36) 

n ——oo 

Plancherel’s relation: 

V x[n\y[n]* = ^- f % X(w)Y((ti)*da> (3.37) 

n =^oc ^ J-n 

We have met two kinds of convolution so far and one more is familiar from continuous-time systems. For 
functions of a continuous variable we have ordinary or linear convolution involving an integral from —°o to 
oo. Flere we have seen periodic or circular convolution of periodic functions of a continuous variable (in this 
case the periodic frequency responses of discrete-time systems) involving an integral over one period. For 
discrete-time systems, we have seen the ordinary or linear convolution which involves a sum from —°o to 
Later in the study of DFTs, we will meet the periodic or circular convolution of sequences involving a sum 
over one period. 
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3.4 Frequency analysis of LTI systems 

LTI system (recap): 


x[fl] 


H{.} 


y[n\ 

—► 


y[n] = x[n] * h[n] = Y x[k\h[n — k\ (3.38) 

k——oo 

h[n\ = 9{{h[n\} (3.39) 

Response to complex exponential: 

• Let x[n] = ei w \ n £Z. 

• Corresponding output signal: 

y[n ] = ^h[k]x[n — k] 
k 

= Y,h[k}e Mn ~ k) 

k 

= {^h[k}e^ ak }e ju>n = H((0)x[n] (3.40) 

k 

where we recognize H( co) as the DTFT of h[n\. 

• Eigenvector interpretation: 

- x[n] = e J(l)n behaves as an eigenvector of LTI system ‘.H\ iHx = he 

- Corresponding eigenvalue X = //(to) provides the system gain 


Definition: 

Consider an LTI system with impulse response /?[«]. The frequency response of T/) denoted //(co), is 
defined as the DTFT of h[n]: 

oo 

H (co) = Y, h[n\e- jwn , co G R (3.41) 


Remarks: 

• If DT system 4/ is stable, then the sequence h[n] is absolutely summable and the DTFT converges 
uniformly: 

- //(co) exists and is continuous. 
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• H{ co) known =>■ we can recover h\n\ from the inverse DTFT relation: 

h[n) = — T H((o)e j(0n d(0 (3.42) 

2n J—% 

• We refer to |//(co) | as the magnitude response of the system. 

We refer to Z//(co) as the phase spectrum. 


Properties: 


Let ‘M be an LTI system with frequency response //(co). Let y[n] denote the response of Of to an arbitrary 
input x[n\. We have 


T(go) = //(co)X(co) 


(3.43) 


Proof: 


Of LTI =>■ y = h*x =0 Y( co) = //(co)X(co) 


Interpretation: 


Recall that input signal x[n] may be expressed as a weighted sum of complex exponential sequences 
via the inverse DTFT: 

x[n] = 2- ^ X(d))e jw, dm (3.44) 

2tz J—ji 


According to (3.43), 


y[n] = — / H((£>)X((ti)e-i w 'd(i) 
2n J-n 


(3.45) 


• Note filtering role of//(co): each frequency component in x[n\, i.e. A(co)e /0, ' ! , is affected by //(co) in 
the final system output y[n ]: 

- gain or attenuation of j//(co)| 

- phase rotation of Z//(co). 


Example 3.8: 

► Consider a causal moving average system: 


• Impulse response: 


y[n\ 


h[n] 


i M— 1 

m k=o 


M -1 


M ^ 


k =0 


M 0<n<M 
10 otherwise 

u[n]-u[n-M ]) 


(3.46) 
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• Frequency response: 

i Af-l 

»(">) = s E 

m n =0 

1 _ i 

" M e ~ jm - 1 

1 g -;'coM/2 e ~jmM/2 _ £ jaM/2 

~ M e -j<°/ 2 e ~ jm/2 - e ja/2 
= 1 g—jio(M—i)/2 sin (coM/2) 

M sin(co/2) 

• The magnitude response is illustrated in Figure 3.5. 

- zeros at to = 2jt, k/M ( k ^ 0), where = ® 

- level of first sidelobe « — 13dB 

• The phase response is illustrated in Figure 3.4. 

- negative slope of — (M — l)/2 

- jumps of Tt at © = 2nk/M ( k ^ 0), where changes its sign. 


(3.47) 


◄ 


z x(0 



Fig. 3.4 Phase response of the causal moving average system (M = 6) 


Further remarks: 


• We have seen that for LTI system, Y (to) = H((o)X (go): 

- X(co) = 0 at a given frequency to =4> Y (go) = 0 
if not: system is non-linear or time-varying 

- //(go) = 0 => Y (go) = 0, regardless of input 


• If system is LTI, its frequency response may be computed as 


//(co) 


T(co) 

X(go) 


(3.48) 
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IX(e |lu )l 



Fig. 3.5 Magnitude Response of the causal moving average system (M = 6) 

3.5 LTI systems characterized by LCCDE 

LCCDE (recap): 

• DT system obeys LCCDE of order N if 

N M 

Y a k y[n ~k] = Y b k x i n ~ k ] 

k =0 k =0 

where ao / 0 and a.y / 0. 

• If we further assume initial rest conditions, i.e.: 

x[n] = 0 for n < n 0 => y[«] = 0 for n < no 

LCCDE corresponds to unique causal LTI system. 


Frequency response: 

Taking DTFT on both sides of (3.49): 


N M 

Y auy[n - fc] = £ b k x[n - k\ 


k =0 


k= 0 


N M 

=► Y a k e~^ k Y{ co) = £ b k e~^ k X{ to) 


k=0 

=4 //(to) 


y(«>) 

X(co) 


k=0 

Ltobke-W 
Lto a k e-j* k 


(3.49) 


(3.50) 


(3.51) 


© 5 . Champagne & F. Labeau 


Compiled September 13, 2004 







44 


Chapter 3. Discrete-time Fourier transform (DTFT) 


3.6 Ideal frequency selective filters: 

Ideal low-pass: 


H lp { co) = 

hLp[n ] 


{ 1 if 1001 < CO c 
0 if CO c < |to| < 71 

©c . , ©c« s 

= — smc( — 

71 K 


(3.52) 


H(co) 


-71 -0) 


Figure 3.6: Low-pass filter 


Ideal high-pass: 


Hnp(m) = 






H(co) 

F 

if C0 c < 00 < 71 


1 


1 ° 

if CO| < 00c 





(3.53) 


-n — CO r 


Figure 3.7: Fligh-pass filter 


Ideal band-pass: 


Hbp( ©) 


1 if |(D — co 0 | < B/2 
0 else in [—71,7i] 


h B p[n\ = 2cos(cd 0 77)/?lpH|co £ .=b/2 


(3.54) 




H(co) 


1 


B 






71 

-co 0 


(Oq 


Figure 3.8: Band-pass filter 


Ideal band-stop: 


Ffes(©) 


0 if |co — C 0 o | < B/2 
1 else in [—71,7i] 


hss[n] = 8[n] - h B p[n\ 


(3.55) 


1 

H(a>) 




, B , 


— 71 — O) o (0 o 71 (O 


Figure 3.9: Stop-pass filter 


Remarks: 

• These filters are not realizable: 

- they are non-causal (h[n\ / 0 for n < 0) 

- they are unstable (£„ \h\n]\ = °°) 


© B. Champagne & F. Labeau 


Compiled September 13, 2004 
































3.7 Phase delay and group delay 


45 


• The main object of filter design is to derive practical approximations to such filters (more on this 
later...) 


3.7 Phase delay and group delay 
Introduction: 

Consider an integer delay system, for which the input-output relationship is 

y[n]=x[n — k\, k € Z. (3.56) 

The corresponding frequency response is computed as 

P (go) = e~ jak X((o) => //(to) = = e~ jwk . (3.57) 

a(co) 

This clearly shows that the phase of the system ZH(c o) = — CO k provides information about the delay incurred 
by the input. 

Phase delay: 

For arbitrary //(go), we define the phase delay as 

A ZH((0) 

nh — 

1 CO 

For the integer delay system, we get: 

ZH( go) _ go/: 

GO GO 

The concept of phase delay is useful mostly for wideband signals (i.e 
to K). 


(3.58) 

(3.59) 

. occupying the whole band from — Jt 


Group delay: 


Definition: 

dZH( go) 
c/go 

This concept is useful when the system input x[n] is a narrowband signal centred around C0 q, i.e.: 


X gr — 


(3.60) 


x[n] = s[n]e j(0on 


(3.61) 


where s[n\ is a slowly-varying envelope. An example of such a narrowband signal is given in Figure 3.7. 
The corresponding system output is then 

y[n] « |//(go ( ,)| 4« - x gr (GOo)]^ COo[ ”- T '"' (c0o)1 . (3.62) 
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Fig. 3.10 An example of a narrowband signal as defined by x[n] = s[n] cos(ft)on). Top: the 
signal x[n\ is shown, together with the dashed envelope s[n] (Samples are connected to ease 
viewing). Bottom: the magnitude of the DTFT of x[n\. 
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The above equation shows that the phase delay x p h{ cc>o) contributes a phase change to the carrier e /0, °", 
whereas the group delay x gr (00o) contributes a delay to the envelope s[n]. Notice that this equation is strictly 
valid only for integer values of x gr ((Oo), though a non-integer value of x gr (<»o) can still be interpreted as a 
non-integer delay 1 . 


Pure delay system: 

• Generalization of the integer delay system. 

• Defined directly in terms of its frequency response: 

H(q a) = e~ jwz 


where x G M is not limited to take integer value. 
• Clearly: 


//(co)| = 1 => no magnitude distortion 

Xph{ CO) = x gr (co) = X 


(3.63) 


(3.64) 

(3.65) 


Linear phase: 

Consider an LTI system with 

H(co) = \H(co)\e jZH{(o) . (3.66) 

If ZH (to) = —cox, we say that the system has linear phase. A more general definition of linear phase will 
be given later, but it is readily seen that a linear phase system does not introduce phase distortion, since its 
effect on the phase of the input amounts to a delay by x samples. 

One can define a distortionless system as one having a gain and a linear phase, i.e. H( oo) = Aexp(j cox). The 
gain is easily compensated for and the linear phase merely delays the signal by x samples. 

3.8 Problems 

Problem 3.1: Ideal High-Pass Filter 

Let h[n] be the impulse reponse of an ideal high-pass filter, i.e. H( co) is defined as: 

, , _ f 0 |co| < co c 

W \ 1 elsewhere ’ 

for the main period —71 < co <n, and for somr given co c , called the cut-off frequency.. 

1. plot//(co). 

2. compute h[n] by using the definition of the inverse DTFT. 

'For instance, a delay of i sample amount to replacing x[n\ by a value interpolated between x[n] and x[n — 1]. 
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3. Given that H{ or) can be expressed as H (or) = 1 —Hu>( or), where HpP( to) is the freqquency response 
of an ideal low-pass filter with cut-off frequency or c , compute h\n\ as a function of hpp [n\. 

4. What would be the frequency response G(to) if g [n] was h\n\ delayed by 3 samples ? 

5. Is the liter h[n\ stable, causal, FIR, HR ? 


Problem 3.2: Windowing 

Let x[n\ = cos(ct)o n) for some discrete frequency Wo, such that 0 < C0o < Jt. Give an expression of X(to). 
Let w[n] = u[n] — u\n—M], Plot the sequence w\n\. Compute IV (or) by using the definition of the DTFT. 
Let now y[n\ = x[n]w[n]. Compute the frequency response Y (or). 

Problem 3.3: 

Given a DTFT pah - 

cr 

x[n] <-> X(oo), 

1. Compute the DTFT of (—l)"jc[n]; 

2. Compute the DTFT of (—1 ) n x*[M — n] for some MG Z; 

3. What is the equivalent in the time -domain of the following frequency domain condition 

|X(C0)| 2 +|X(7t-0T)| 2 = 1? 

[Hint: Recall that \a\ 2 = a*a .] 
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Chapter 4 

The z-transform (ZT) 


Motivation: 

• While very useful, the DTFT has a limited range of applicability. 

• For example, the DTFT of a simple signal like x[n\ = 2 "u[n\ does not exist. 

• One may view the ZT as a generalization of the DTFT that is applicable to a larger class of signals. 

• The ZT is the discrete-time equivalent of the Laplace transform for continuous-time signals. 


4.1 The ZT 


Definition: 


The ZT is a transformation that maps DT signal x[n] into a function of the complex variable z, defined as 


oo 

x (z) = L x \ Ti \ z ~ n 

n =—co 


(4.1) 


The domain of X(z) is the set of all z G C such that the series converges absolutely, that is: 

Dom(X) = {z E C : ^ < °°} (4-2) 

n 


Remarks: 

• The domain of X(z) is called the region of convergence (ROC). 

• The ROC only depends on |z|: if z £ ROC, so is ze'^ for any angle (|>. 

• Within the ROC, X(z) is an analytic function of complex variable z. (That is, X(z) is smooth, derivative 
exists, etc.) 


Both X(z) and the ROC are needed when specifying a ZT. 
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Example 4.1: 

► Consider a [h] = 2"u[n]. We have 



where the series converges provided |2 z 1 1 < 1. Accordingly, ROC : \z\ >2 


Connection with DTFT: 


• The ZT is more general than the DTFT. Let z = re /0) , so that 


oo 


ZT {x[n]}= Y x[n\r- n e- jan = DTFT{x[n\r~ n } 


(4.3) 


With ZT, possibility of adjusting r so that series converges. 

• Consider previous example: 

- x[n] = 2 n u\n\ does not have a DTFT (Note: £„ \x[n] | = °°) 

- x[n] has a ZT for |z| = r > 2 

• If ^ = e-t® c ROC, 


X(e JW ) = Y Me~ jw = DTFT{x[n]} 


(4.4) 


• In the sequel, the DTFT is either denoted by X(e J<>y ), or simply by X(co) when there is no ambiguity 
(as we did in Chapter 3). 

Inverse ZT: 

Let X(z), with associated ROC denoted be the ZT of DT signal x{n\. Then 



n€ Z 


(4.5) 


where C is any closed, simple contour around z = 0 within ‘I{y. 

Remarks: 

• In analogy with the inverse DTFT, signals z n , with relative weight A(z). 

• In practice, we do not use (4.5) explicitly to compute the inverse ZT (more on this later); the use 
of (4.5) is limited mostly to theoretical considerations (e.g. next item). 

• Since DT signal x[n] can be recovered uniquely from its ZT X(z) (and associated ROC %), we say 
that x[n] together with X(z) and ^ form a ZT pair, and write: 


x[n] 


X(z),ze^ 


(4.6) 
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4.2 Study of the ROC and ZT examples 

Signal with finite duration: 

• Suppose there exist integers /V| < /V? such that x\n] = 0 for n < N\ and for n > AS. Then, we have 

n 2 

x (z) = £ x[n]z~ n 

n—N\ 

= x\Ni]z~ Nl +x[Ni + + ■ ■ ■ +x[N 2 }z~ N2 ( 4 . 7 ) 

• ZT exists for all z G C, except possibly at z = 0 and z = °°- 

- N 2 >0^z = 0(jt. ROC 

- N!<0^z = °°(£ ROC 


Example 4.2: 


► Consider the unit pulse sequence x[n] = 8[n]. 

X(z) 

ROC 


1 x z° = 1 
C 


(4.8) 

◄ 


Example 4.3: 

► Consider the finite length sequence x[n] = {1,1,2,1}. 

X(z) = z+l+2z -1 +z -2 

ROC : 0 < |z| < °° (4.9) 

◄ 


Theorem: 

To any power series Yf~o c n w '\ we can associate a radius of convergence 


R = lim 

n—>°o 


Cji 


Cn +1 


(4.10) 


such that 


if |w| < R =s the series converges absolutely 
if |w| > R =>■ the series diverges 
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Causal signal: 

Suppose x[n] = 0 for n < 0. We have 

oo oo 

x (z) = Y x W~ n = Y c ' iVv " 

71=0 71=0 

c„=x[n], w = z _1 

Therefore, the ROC is given by 

M 

\z\ 

The ROC is the exterior of a circle of radius r, as shown in Figure 4.1. 


I>r 

~Re( z ) 



< R w = lim 

71—^o° 


x\n\ 


x[n+ 1] 


> Rw r 


Fig. 4.1 Illustration of the ROC for a causal signal 


Example 4.4: 

► Consider the causal sequence 


x[n\ = a n u[n] 

where a is an arbitrary complex number. We have 


X(z) 




72=0 


72=0 


1 

1 — az ~ 1 


(4.13) 


(4.14) 


(4.11) 


(4.12) 
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provided |az 1 1 < 1, or equivalently, |z| > \a\. Thus, 

ROC : |z| > \a\ (4.15) 

Consistent with Figure 4.1, this ROC corresponds to the exterior of a circle of radius r = \a\ in the 
Z-plane. ◄ 


Anti-causal signal: 

Suppose jc[»] = 0 for n > 0. We have 

0 oo 

*(z)= £ x[n]z~ n = Y, x[~ n V 

n=—<x> n—0 

Therefore, the ROC is given by 

|z| < R = lim 

n-^ oo 

The ROC is the interior of a circle of radius R in the z-plane, as shown in Figure 4.2. 


x[—n] 
x[—n — 1] 



Fig. 4.2 Illustration of the ROC for an anti-causal signal. 


(4.16) 


(4.17) 


Example 4.5: 

► Consider the anti-causal sequence 


x\ri\ = —a n u[—n — 1]. 


(4.18) 
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We have 


X(z) = 


E a " z ~ n 


-E(«-^r 

n= 1 

1 — 

1 


if I a 1 z| < 1 


1 — az 1 


provided |a J z| < 1, or equivalently, |z| < \a\. Thus, 


ROC : |z| < Ifll 


(4.19) 


(4.20) 


Consistent with Figure 4.2, the ROC is the interior of a circle of radius R = \a\. Note that in this and the 
previous example, we obtain the same mathematical expression forX(z), but the ROCs are different. ◄ 


Arbitrary signal: 

We can always decompose the series X(z) as 

x (z)= E *Mz~"+E*Mz~" 

n=—°° n =0 

V -v-' '-V-" 

needs |z|>r needs 

We distinguish two cases: 

• If r < R , the ZT exists and ROC : r < |z| < R (see Figure 4.3). 



• If r > R, the ZT does not exist. 


(4-21) 
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Example 4.6: 

► Consider 

x[n] = (1 /2) n u[n] — 2 "u[—n — 1]. 

Here, we have 

*(z) = £(l/2)"z“' ! - £ 2 n z~ n 



|ikl>l/2 |ikl<2 

1 | 1 

1-^z” 1 + 1 —2z~ 1 



The two series will converge simultaneously iff 

ROC: 1/2 < |z| <2. 

This correspond to the situation shown in Figure 4.3 with r = 1/2 and R 2. 


(4.22) 


(4.23) 


(4.24) 

◄ 


Example 4.7: 

► Consider 

x[n] = 2”w[n] — (1/2 )™m[— n— 1]. (4.25) 

Here, X (z) = X + (z) + (z) where 

X + (z) = £ 2”z“" converges for |z| > 2 

n—0 

-1 

X-(z.) = £ (l/2)”z _,! converges for |z| < 1/2 

n=—o o 

Since these two regions do not intersect, the ROC is empty and the ZT does not exist. ◄ 

4.3 Properties of the ZT 

Introductory remarks: 

• Notations for ZT pans: 

x[n] X(z), j 

y[n) ^ T(z), ze% 

and % respectively denote the ROC of X (z) and Y (z) 

• When stating a property, we must also specify the corresponding ROC. 

• In some cases, the true ROC may be larger than the one indicated. 
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Basic symmetries: 


x[-n] X(z '), Z 1 <E ‘Kx 

x*[n\ ^ 

Linearity: 

ax[n\ + by[n ] aX(Z) + bY(z), 


(4.26) 

(4.27) 


(4.28) 


Time shift (very important): 


x[n-d]<r-^z d X(z), zC %i 


(4.29) 


Exponential modulation: 


a"x[n] <— >X(z/ci), z/aC% 


(4.30) 


Differentiation: 


z dX(z) 

nx\n\ <—> —z —;—, z^A)- 


dz 


(4.31) 


Convolution: 


x[n]*y[n\ <-^X(z)Y(z), 


(4.32) 


Initial value: 

For x[n] causal (i.e. x\n] = 0 for n < 0), we have 

limX(z)=x[0] (4.33) 

z—>°° 
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Example 4.8: 

► Consider 


We have 


x\n] = cos(co 0 h)m[m] 

= ie^ n n[n] + ^e~ lahn u[n] 


m 


{ -ZT{e i(a ° n u[n}} + ^ZT{e~ jahn u[n]} 

11 11 

2 1 —eJ <0 °z~^ 2 1 — e~l <0 °z~^ 


|z|>|^’®o| = l |z| 

1 — z~ 1 cos C 0 o 

1 — 2z~ 1 COS (0 o + Z~ 2 ’ 


\e-i«to\=l 

ROC : \z\ > 1 


Example 4.9: 

► Consider 

We have 


x\n\ = na u\n\ 


x{z) = 

az 


z > a 


t/z \ 1 — az 1 J ’ 

(1 — az~ x ) 2 ’ ROC: l z l > l a l 


◄ 


◄ 


Example 4.10: 

► Consider the signals *1 [n] = {j_, —2a,a 2 } and X 2 [n] = {l,a,a 2 ,a 3 ,a 4 } where a £ C. Let us compute the 
convolution y = x 1 *x 2 using the z-transform: 

X\ (z) = 1 -2az -1 +a 2 z~ 2 = (1 — az -1 ) 2 

X 2 (z) = 1 +az~' +a 2 z _2 + a 3 z _3 + fl 4 z~ 4 = ^— ° " . 

1 — az 1 


Y(z) = Xi(z)X 2 (z) = (1 -az l )(l-a 5 z 5 ) 
= 1 —az~ 1 —a 5 z~ 5 +a 6 z~ 6 

Therefore, y[n] = {1, — a,0,0,0, — a 5 ,a 6 }. 


◄ 
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4.4 Rational ZTs 


Rational function: 


X (z) is a rational function in z (or z 1 ) if 



(4.34) 


where N(z) and D(z) are polynomials in z (resp. z 1 ) 

Remarks: 

• Rational ZT plays a central role in DSP 

• Essential for the realization of practical HR filters. 

• In this and the next Sections, we investigate two important issues related to rational ZT: 

- Pole-zero (PZ) characterization 

- Inversion via partial fraction expansion 

Poles and zeros: 

• X (z) has a pole of order L at z = p 0 if 


*( Z ) = ( z _ po )L » °<MPo)\<°° 


(4.35) 


• X (z) has a zero of order L at z = z 0 if 


X(z) = (z~Zo) L y{z), 0 < |V|/(z 0 )| < oo 


(4.36) 


• We sometimes refer to the order L as the multiplicity of the pole/zero. 


Poles and zeros at OOJ 


• X (z) has a pole of order L at z = °o if 


X(z) = z L \ |/(z), 0 < |\|/(°o) | < oo 


(4.37) 


• X (z) has a zero of order L at z = 00 if 



(4.38) 
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Poles & zeros of a rational X (z): 

Consider rational function X (z) = N(z) /D(z ): 

• Roots of N(z) =>■ zeros of X (z) 

Roots of D(z) => poles of X(z) 

• Must take into account pole-zero cancellation: 

common roots of N(z) and D(Z) do not count as zeros and poles. 

• Repeated roots in N(z) (or D(z)) lead to multiple zeros (respectively poles). 

• If we include poles and zeros at 0 and °o; 


number of poles = number of zeros 


(4.39) 


Example 4.11: 


► (1) Consider the rational function 


z 2 — 2z+l (z— l) 2 

The poles and zeros of X(z) along with their order are as follows: 



poles : p i = l, L = 2 
zeros : zi = 0, L = 1 


Z2 = °°, L= 1 


(2) Let 



The corresponding poles and zeros are 


zeros : Zk = e 2%k ^ 2 for k = 0,1,2,3, L=1 


poles : pi= 0, L = 3 (triple pole) 
P2 = —3, L= 1 


(3) As a final example, consider 


X(z)=z- 1 


The poles and zeros are 


zeros : zi = 1, L= 1 
poles : pi = oo, L = 1 
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Pole-zero (PZ) diagram: 

For rational functions X(z) = N(z)/D(Z), knowledge of the poles and zeros (along with their order) com¬ 
pletely specify X(z), up to a scaling factor, say G G C. 


Example 4.12: 

► Let us consider the following pole and zero values: 


zi = 1 

Pi =2 


L= 1 
L= 1 



X(z) = G 


z- 1 
z —2 


= G 


1-z- 1 
1-2 z - 1 


(4.40) 

◄ 


Remarks: 

• Thus, we may represent X(z) by a so-called PZ diagram, as illustrated in Figure 4.4. 


4k). 


t lm(z) 


pole of order k 


o( k )=zero of order k 


x 


x 


,( 2 ) 



Re(z) 



Fig. 4.4 Example of a PZ diagram. 

• For completeness, the presence of poles or zeros at °o should be mentioned on the diagram. 

• Note that it is also useful to indicate ROC on the pz-diagram. 

Example 4.13: 

► Consider a[«] = a n u\n], where a > 0. The corresponding ZT is 

X(z)= , 1 , = —, ROC :\z\> a (4.41) 

1 — az 1 z —a 

Zi — 0 , L = 1 
pi= a , L = 1 

The PZ diagram is shown in Figure 4.5. ◄ 
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Fig. 4.5 PZ diagram for the signal x[n\ = a n u[n]. 


ROC for rational ZT: 

Let’s summarize a few facts about ROC for rational ZTs: 

• ROC does not contain poles (because X(z) does not converge at a pole). 

• ROC can always be extended to nearest pole 

• ROC delimited by poles =>- annular region between poles 

• If we are given only X(z), then several possible ROC: 

- any annular region between two poles of increasing magnitude. 

- accordingly, several possible DT signals x[n] 

4.5 Inverse ZT 

Introduction: 

• Several methods do exist for the evaluation of x[n\ given its ZT X(z) and corresponding ROC: 

- Contour integration via residue theorem 

- Partial fraction expansion (PFE) 

- Long division 

• Partial fraction is by far the most useful technique in the context of rational ZTs 

• In this section: 

- we present the PFE method in detail; 

- we discuss division only as a useful tool to put X (z) in proper form for the PFE. 
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4.5.1 Inversion via PFE 


PFE: 


• Suppose that X(z) = where 

- N(z) and D(z) are polynomials in z~ l 

- degree of D(z) > degree of N(z) 

• Under these conditions, X(z) may be expressed as 



fc=t/=t 


Am 

(1 ~PkZ~ l Y 


where 

- pi,..., pk are the distinct poles of X(z) 

- L \,... ,Lk are the corresponding orders 


• The constants Am can be computed as follows: 

- simple poles (L k = 1): 

Am =A k i = (1 ~ PkZ~ l )X{z)\z= Pk 

- multiple poles (L k >1): 


Am 


1 ^^-‘[(l-pkZ-^Xjz)} ,, 

(L k — l)\(—pk) Lk ~ l ^ (dz~ l ) Lk ~ l *'z=Pk 


(4.42) 


(4.43) 


(4.44) 


Inversion method: 

Given X(z) as above with ROC : r < |z| < R, the corresponding DT signal x[n] may be obtained as follows: 
• Determine the PFE of X (z ): 


K 4 

iwilTTT 1 )' 


*M = EEtt A " 


Invoking linearity of the ZT, express x[n] as 

K L k 


x\n\ 


k=ll=\ 

where Z ~ 1 denotes the inverse z-transform. 

Evaluate the elementary inverse ZTs in (4.46): 
- simple poles (L k =1): 

1 z 


= h A klZ -Tly} 

,, l (i -Pkz 


if \Pk\ < r 
if \pk\ >R 


(4.45) 


(4.46) 


(4.47) 
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higher order poles (L* >1): 


1 2T 1 I ("}_ 1 )p"k u H if Iwl < r 

(l-pr-z -1 )' } — « — !] ii\Pk\>R 


where (") = r!( ”y r) , (read n choose r) 


(4.48) 


Example 4.14: 

► Consider 

* (;)= (I -«-■)’(!-fe-') ’ w < |z| < M 

The PFE of X(z) can be rewritten as: 

4, , A 2 


with 


so that 


X(z) = 


1 — az 1 1 — bz 1 ’ 


A! = (l-« ‘)^)U= ^ 


A 2 = (l-bz- l )X(z)l_ b = 

\z-o l— az 1 


z—b 


a — b 


b — a 


+1 = 




|z|>|<3|=>causal |z|<|&|=>anti-causal 

= A\a n u[n] — A 2 £>"n [—n — 1] 

„n +1 / 9 «+l 

u[n] — - - u[-n-\] 


a—b b—a 


4.5.2 Putting X (z) in a suitable form 
Introduction: 

• When applying the above PFE method to X(z) = N(z ) /D(z), it is essential that 

- N(z) and D(z) be polynomials in z~ l 

- degree of D(z) > degree of N(z) 

• If either one of the above conditions are not satisfied, further algebraic manipulations must be applied 
to X (z) 

• There are two common types of manipulations: 

- polynomial division 

- use of shift property 
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Polynomial division: 

• Find Q(z) and R(z), such that 

_ 




m 


D(z) 


(4.49) 


where Q(z) = Ym=Ni x i n \ z " an d R(z) * s a polynomial in z 1 with degree less than that of D(z) 
Determination of Q(z) and R(z) via a division table: 

Q(z) 


D(z) 


N(z) 

—Q{z)D{z) 


(4.50) 


R(z) 


• If we want the largest power of z in N(z) to decrease, we simply express D(z) and N(z) in decreasing 
powers of z (e.g. D(z) = 1+2z _1 +z~ 2 ) 

• If we want the smallest power of z in N(z) to increase we write down D(z) and N(z) in reverse order 
(e.g. D(z) =z' 2 + 2z _1 + 1) 


Example 4.15: 


► Let us consider the z-transform 


*(z) = 


-5 + 3 z 1 +z 


3 + 4z 1 + z 2 

with the constraint that x[n] is causal. The first step towards finding x[n] is to use long division to make 
the degree of the numerator smaller than the degree of the denominator. 1 


1 

z _2 + 4 z _1 +3 | z =T +3z =rr ^-5~ 

— (z _ 2 + 4 z _1 + 3 ) 
- z 1 8 


( 4 . 51 ) 


so that X(z) rewrites: 


X(z) = 1 - ■ 


! +4z- ] +3 


The denominator of the second term has two roots, the poles at z = -' and z = — 1, hence the factoriza¬ 


tion: 


*(z) = l-x- 


8 


3(l + iz- 1 )(l+z- 1 )' 

The PFE of the rational term in the above equation is given by two terms: 


1 


x(;) = l-3 


'1 


A 2 


1 + jZ -1 


'That is, we want the smallest power of z in N(z) to 
reverse order when performing the division. 


1+z- 1 J ’ 

increase from zT~ to z - '. Accordingly, we write down N(z) and D(z) in 
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with 

(4.52) 

(4.53) 

so that 

js-U) = 1 + ( 6( i + 1^-1) - z(TTi=iy) - 

Now the constraint of causality of x[n\ determines the region of convergence of X (z), which is supposed 
to be delimited by circles with radius 1/3 and/or 1. Since the sequence is causal, its ROC must extend 
outwards from the outermost pole, so that the ROC is kl > !■ The sequence x[n] is then given by: 

M = S[n] + !(4)"«M-\(-!)"«[«]■ 


A] = 


A 2 = 


z -1 +8 
1 +z _1 


-l 


-8 


1 + iZ 


U-l 


z—— 1 


5 

~2 

21 

~2' 


Use of shift property: 

• In some cases, a simple multiplication by z k is sufficient to put X(z) into a suitable format, that is: 

F(z)=z*X(z) = ® (4.54) 

where N(z ) and D[z) satisfy previous conditions 

• The PFE method is then applied to Y (z), yielding a DT signal y[n] 

• Finally, the shift property is applied to recover x[n \: 

x[n]=y[n—k] (4.55) 


Example 4.16: 

► Consider 

l-z“ 128 

x (z) = -j \z\ > 1 

1 —z - 

We could use division to work out this example but this would not be very efficient. A faster approach 
is to use a combination of linearity and the shift property. First note that 

X(z) = F(z)-z- 128 F(z) 

where 

1 1 

Y{Z) ~ T^F 2 “ (l-z“ 1 )(l+z- 1 ) 

The inverse z-transform of F (z) is easily obtained as (please try it) 

F["] = ^(l+ (-!)>["] 
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Therefore 

x[n\ = y[n] —y[n — 128] = - (1 + (— l)")(w[n] — u[n — 128]) 

◄ 


4.6 The one-sided ZT 

Definition 


X + (z) = Z + {x[n ]} = £ x[n]z " 

n— 0 

• ROC: | z| > r, for some r > 0 

• Information about x[n] for n < 0 is lost 

• Used to solve LCCDE with arbitrary initial conditions 

Useful properties: 

• Time shift to the right (k > 0): 

x[n — k] ^ z- k X+(z)+x[-k] +x[-k + l\z~ 1 +■■■ +^[-l]z-^- 1} 

• Time shift to the left (k > 0): 

x[n + k] ^ z k X + (z) -x[0}z k - x[l\z k ~ { - x[k — l]z 


Example 4.17: 

► Consider the LCCDE 

y[n] = ay[n — 1] +-r[n], n> 0 (4.59) 

where x[n] = (3 u[n], a and (3 are real constants with a -/=■ 1 and y[—1] is an arbitrary initial condition. Let 
us find the solution of this equation, i.e. y[n\ for n > 0, using the unilateral ’-transform Z> + . 

• Compute Z + of x\n\. 

x[n ] causal =»X + (z) =X(z)= - —^- 77 , |z| > 1 

1 — z 


(4.56) 


(4.57) 


(4.58) 


© B. Champagne & F. Labeau 


Compiled September 13, 2004 





4.7 Problems 


67 


• Apply Z + to both sides of (4.59) and solve for T + (z): 


where 


r + (z) 


=>(l-az _ 1 )T + (z) 

=>r+(z) 


a(z- 1 y + (z)+y[-l])+A+(z) 

az~ 1 Y + (z) + ay [-1] + ^ , 

1 — z 

«>'[-!] + 

«y[-i] _ P _ 

l-az^ 1 (1 -ocz _1 )(l-z _1 ) 

«y[-!] | A | b 

1 —az^ 1 1 — az” 1 1—z _1 


A = 


ap 
1 -a 


B = 


P 

1 - a 


• To obtain y[n] for n > 0, compute the inverse unilateral ZT. This is equivalent to computing the 
standard inverse ZT under the assumption of a causal solution, i.e. ROC : |z| > max(l, |a|). Thus, 
for n > 0 


y[«] = ay[—l]a"u[n]+Aa"u[n\+B(l)"u[n] 

= y[-iK + 1 +p( 1 f^), »>o 


◄ 


4.7 Problems 


Problem 4.1: Inverse ZT 

Knowing that h\n] is causal, determine h[n\ from its z-transform H(z)'- 


m 


2 + lz- 1 

(l + ^- 1 )(l -kz- 1 )' 


Problem 4.2: 

Let a stable system H(z) be described by the pole-zero plot in figure 4.6, with the added specification that 
H(z) is equal to 1 at z = 1. 

1. Using Matlab if necessary, find the impulse response h[n\. 

2. From the pole-zero plot, is this a low-pass, high-pass, band-pass or band-stop filter ? Check your 
answer by plotting the magnitude response |//(co)|. 


Problem 4.3: 

Let a causal system have the following set of poles and zeros: 
• zeros: 0, |c ± - /It,/4 ,0.8; 
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Fig. 4.6 Pole-Zero Plot for problem 4.2. 
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• poles: ±|, Ig^' 71 / 6 . 

It is also known that //(z)| z =i = 1. 

1. Compute H(z) and determine the Region of Convergence; 

2. Compute h[n\. 

Problem 4.4: Difference Equation 

Let a system be sepcified by the following LCCDE: 



with initial rest conditions. 

1. What is the corresponding transfer function H (z)? Also determine its ROC. 

2. Compute the impulse response h[n] 

(a) by inverting thez-transform H (z); 


(b) from the LCCDE. 


3. What is the output of this system if 



(b) x[n] = 8[n — 3]. 


© B. Champagne & F. Labeau 


Compiled September 13, 2004 




70 


Chapter 5 

Z-domain analysis of LTI systems 


5.1 The system function 

LTI system (recap): 


y[n] 

= x[n\ *h[n\ = ^ x[k\h[n — k\ 

k ——oo 

(5.1) 

h[n] 

= nm 

(5.2) 


Response to arbitrary exponential: 

• Let x[n] = z", n G Z. The corresponding output signal: 

W{z"} = 'L.hW- 1 

k 

= {£ h[k]z- k }z" = H(z)z n (5.3) 

k 

where we recognize H(z) as the ZT of h[n\. 

• Eigenvector inteipretation: 

- x\n] = z" behaves as an eigenvector of LTI system 9~C\ ‘J-fx = Xx 

- CoiTesponding eigenvalue X = H(z) provides the system gain 


Definition: 

Consider LTI system ff{ with impulse response h\n]. The system function of denoted H(z), is the ZT of 
h[n]: 

oo 

H(z)= £ h[n]z~ n , z €. ‘Kh (5.4) 


where ‘I(n denotes the corresponding ROC. 





5.2 LTI systems described by LCCDE 
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Remarks: 

• Specifying the ROC is essential: two different LTI systems may have the same H(z) but different 

• If H(z) and ‘J{u are known, h\n] can be recovered via inverse ZT. 

• If z = <? 7 “ £ i.e. the ROC contains the unit circle, then 

oo 

H(e ja )= £ h[n}e- jan = H{ to) (5.5) 

77——co 

That is, the system function evaluated at z = c 7C0 corresponds to the frequency response at angular 
frequency to. 


Properties: 

Let Of be LTI system with system function H(z) and ROC %u. 

• If y[n] denotes the response of Of to arbitrary input x[n], then 

Y{^=H(z)X(z) 

• LTI system Of is causal iff 1% is the exterior of a circle (including °o). 

• LTI system Of is stable iff 1% contains the unit circle. 


Proof: 


• Input-output relation: 


• Causality: 

• Stability: 


Of LTI => y = h*x => Y{z) = H(z)X(z) 

Of causal h[n\ = 0 for n < 0 : r < |z| < °o 

Of stable <=> ^ \h[n] | =£ \h[n\e~^ n \ < °° e 7<D 6 

n n 


□ 


5.2 LTI systems described by LCCDE 

LCCDE (recap): 

• DT system obeys LCCDE of order N if 

N M 

X a k y[n - k\ = £ hx[n ~ k \ 
k =0 k =0 

where «o f 0 and <x\; f 0. 


(5.6) 


(5.7) 
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• If we further assume initial rest conditions, i.e.: 

x[n] = 0 for n < «o => y[n\ = 0 for n < no 
LCCDE corresponds to unique causal LTI system. 


(5.8) 


System function: 

Taking ZT on both sides of (5.7): 


N M 

Yj a k y[n - k\ = Y b k x[n - k] 


k =o 


k =o 


N 


Y akZ ky ( z ) = 

k =0 


#(z) 


y(z) 


Y J b k z- k X{z) 

k =0 

Ltohz- k 

Lk= 0 a k z- k 


(5.9) 


Rational system: 

More generally we say that LTI system is rational iff 

H(z) = z 

where L is an arbitrary integer and 


A(z) 


N M 

A(z) = £ a k z~ k , B(z) = £ b k z k 
k =o k =o 

Factored form: 

If the roots of B(z) and A(z) are known, one can express H(z) as 

-LUt^-ZkZ- 1 ) 


(5.10) 


(5.11) 


H(z) = Gz 


IlLii'-PkZ- 1 ] 


(5.12) 


where G = system gain (G M or C), z k s are non-trivial zeros (i.e ^ 0 or oo), p k \ are non-trivial poles. Lactor 
z~ L takes care of zeros/poles at 0 or °o. 


Properties: 

• Rational system (5.12) is causal iff ROC = exterior of a circle: 

=>• no poles atz = °o=^L>0 
=> ROC: |z| > max,t \p k \ 

• Rational system is causal and stable if in addition to above: 

=> unit circle contained in ROC, 

=> that is: \p k \ < 1 for all k 
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Rational system with real coefficients: 


• Consider rational system: 


hH _.-lB(z) _.- L Lf =0 b k z- k 

W ' Mz) * Eto a k z~ k 

(5.13) 

• In many applications, coefficients a k s and b k s E M. This implies 


H*{z) = H{z*) 

(5.14) 

• Thus, if Zk is a zero of H(z), then 


H(z* k ) = (H(zk))* = 0* = 0 

(5.15) 


which shows that z* k is also a zero of H{z) 

• More generally, it can be shown that 

- If p k is a pole of order / of H(z), so is p* k 

- If Zk is a zero of order / of H(z), so is z* k 

We say that complex poles (or zeros) occur in complex conjugate pairs. 

• In the PZ diagram of H(z), the above complex conjugate symmetry translates into a mirror image 
symmetry of the poles and zeros with respect to the real axis. An example of this is illustrate in 
Figure 5.1. 


Im(z) 


Pk x 


° ^ 


Pk x 


° z k 


Re(z) 


Fig. 5.1 PZ diagram with complex conjugate symmetry. 

5.3 Frequency response of rational systems 

The formulae: 

• Alternative form of H{z) (note K = L + M — N): 

(5.12) =» H(z) = Gz- K Hl='l Z ~ Zk \ (5.16) 

U k =i(z-Pk) 
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• Frequency response: 


• Define: 


• Magnitude response: 


• Phase response: 


• Group delay: 


_ Cr -iwKll¥=i(e ja -z k ) 

" (®) — li{z)\ z=e j<a Ge N 

Y\L\( e '-Pk) 

(5.17) 

V k ((0) = \e ja -z k \ U k ((o) = \e ja -p k \ 

0 A (co) = Z(e J(a - Zk) <>/t(co) = Z(e J(0 - p k ) 

(5.18) 

1 v 1 't/i((o)...£/jv(co) 

(5.19) 

M N 

\H((0) dB = \G dB + Y Vjfc(C0)|dB — Y ^4( ro )|dB 
k= 1 k=l 

(5.20) 

M N 

ZH( to) = ZG - co K + Y 0*(to) - Y <k(w) 

k= 1 k= 1 

(5.21) 

j M N 

v(®) = = k +'L e *(®) - L ^(®) 

“® k= 1 k= 1 

(5.22) 


Geometrical interpretation: 

• Consider pole p k : 



- A = e /0) — pk : vector joining p k to point e jm on unit circle 

- £4((fl) = |A|: length of vector A 

- <]>*(co) = ZA: angle between A and real axis 

• A similar interpretation holds for the terms 14(co) and 0 A (co) associated to the zeros z k -» 


© B. Champagne & F. Labeau 


Compiled September 13, 2004 










5.4 Analysis of certain basic systems 


75 


Example 5.1: 

► Consider the system with transfer function: 


In this case, the only zero is at z 


H(z) = 


1 


1-.8Z- 1 z— .8 
0, while the only pole is at z = .8. So we have that 


Vi(co) = \e ja> \ = 1 0 1 (co)=<B 

f/i(co) = \e j<0 — .8| (|)i (co) = Z(e ;fi> — .8) 


(5.23) 


The magnitude of the frequency response at frequency co is thus given by the inverse of the distance 
between .8 and e J< " in the z-plane, and the phase response is given by co — 4>i (co). Figure 5.2 illustrates 
this construction process. ◄ 


Some basic principles: 

• For stable and causal systems, the poles are located inside the unit circles; the zeros can be anywhere 

• Poles near the unit circle at p = re JW ° (r < 1) give rise to: 

- peak in |//(co) | near 00 o 

- rapid phase variation near oo 0 

• Zeros near the unit circle at z = re ,co " give rise to: 

- deep notch in |//(co)| near C0 o 

- rapid phase variation near co 0 

5.4 Analysis of certain basic systems 

Introduction 

In this section, we study the PZ configuration and the frequency response of basic rational systems. In all 
cases, we assume that the system coefficients aj-s and bk are real valued. 

5.4.1 First order LTI systems 
Description: 

The system function is given by: 

1 — bzT ^ 

H(z) = G- -p (5.24) 

1 — az~ l 

The poles and zeros are: 

- pole: z = a (simple) 

- zero: z = b (simple) 
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Fig. 5.2 Illustration of the geometric construction of the frequency response of the signal 
in (5.23), for CO = 7t/5 (top) and CO = Jt/2 (bottom) 
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Practical requirements: 

- causality: ROC : |z| > |a| 

- stability: |a| < 1 

Impulse response (ROC: |z| > |a|): 

h[n] = G( 1- )a n u[n] +G-8[n] (5.25) 


Low-pass case: 

To get a low-pass behavior, one needs a = 1 — e, where 0<£C 1 (typically). Additional attenuation of 
high-frequency is possible by proper placement of the zero z = b. 


Example 5.2: Low-Pass first order system 

► Two examples of choices for b are shown below. 



zero: b = 0 

h ‘< z) = c 'T^t 

G\ — 1 — a => H\ (to = 0) = 1 


zero: b = — 1 

H i{z) = G 2 + ~_, 

1 — az 1 

G 2 = ^ ^ ^2 (to = 0) = 1 
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The frequency responses of the corresponding first order low-pass systems are shown in Figure 5.3. ◄ 





Fig. 5.3 Frequency response of first order system 


High-pass case: 

To get a high-pass behavior, one has to locate the pole at a = — 1 + e, where 0 < e < 1 To get a high 
attenuation of the DC component, one has to locate the zero at or near b = 1. 


Example 5.3: High-Pass first order system 

► The PZ diagram and transfer function of a high-pass first order system with a = —.9 and h = 1 are shown 
below. 
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The corresponding frequency response is shown in Figure 5.4 


zero: b = 1 

=>»(-*)= 1 


◄ 




Fig. 5.4 Frequency response of a first-order high-pass filter with a pole at z = —.9 and a zero 
atz = 1 


5.4.2 Second order systems 
Description: 

• System function: 


H{z) = G 


1 +b\z l +b 2 z 2 
\+a\z~ x +a 2 z~ 2 


(5.26) 
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• Poles: 

- if > 4aj'. 2 distinct poles (real) at p\^ = — y 

- if a 2 = 4fl2: double pole (real) at p\ = — ^ 

- ifa?< 4(72: 2 distinct poles (complex) at p\ 2 = 

• Practical requirements: 

- causality: ROC : |z| > max{|p 1 1, |/? 2 1} 

- stability: can show that \pi<\ < l for all k iff 

|«21 < 1, (72>|fll| — 1 (5.27) 


± j \Ja\ — 4(72 

-y ±j\\J^ a 2-a\ 


Resonator: 



Pi 

P2 


— re JU>o 


re j(0 ° = p\ 


H(z) = G 
= G 


1 


(1 — rei a ° z 1 ) (1 — re ^ COoZ 1 ) 

1 

1 — 2 reos(c 0 o )z“ 1 + r 2 z~ 2 


• The frequency response (Figure 5.5) clearly shows peaks around ±co 0 . 

• For r close to 1 (but < 1), |//(ot>)| attains a maximum at to = ±co„ 

• 3dB-bandwidth: for r close to 1, can show: 

|Z/(oo 0 d= ^)| = —\=, where Aco = 2(1 — r) (5.28) 

2 y 2 


Notch filter: 

A notch filter is an LTI system containing one or more notches in its frequency response, as the results of 
zeroes located on (or close to) the unit circle. 
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Fig. 5.5 Frequency response of a resonator system 



Z 1 
Pi 


e J(0 ° , Z 2 = e j(0 ° = z\ 


re 


ja> ° P 2 = re J(0 ° = p\ 


H(z) 


(1 — rei a °z ~ 1 ) (1 — re~J m °z ~ 1 ) 
1 -2cos(co 0 )z _1 +z~ 2 
1 — 2rcos(oo 0 )z _1 +r 2 z ~ 2 


The notches in the frequency response are clearly shown in Figure 5.6. 


5.4.3 FIR filters 
Description: 

• System function: 

H(z) = B{z) = bv + b\z~ l H- Yb M z~ M 

= b 0 (l — Ziz L ) - - - (1 -ZMZ~ l ) (5.29) 
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Fig. 5.6 Frequency response of a notch filter. 


• This is a zeroth order rational system (i.e. A(z) = 1) 
The M zeros Zk can be anywhere in the complex plane 
There is a multiple pole of order M at z = 0. 


• Practical requirement: none 

Above system is always causal and stable 


• Impulse response: 


( b ,, 0 <n<M 
\ 0 otherwise 


Moving average system: 

• Difference equation: 


^ M-l 

>’M = -Tj £ x[n-k] 

m k=0 


• System function: 


H(z) 


1 

M 


M— 1 

L*~ k 


k =0 


1 1 -z' M 
M 1 —z _1 


(5.30) 


(5.31) 


(5.32) 
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• PZ analysis: Roots of the numerator: 

Z M =1 ^z = e j2Kk/M , k = 0, 1, ...,M — 1 (5.33) 

Note: there is no pole at z = 1 because of PZ cancellation. Thus: 

i M—1 

( 5 - 34 ) 

m k= i 

• The PZ diagram and frequency response for M = 8 are shown in Figures 5.7 and 5.8 respectively. 



Fig. 5.7 Zero/Pole diagram for a Moving Average System (M = 8). 


5.5 More on magnitude response 

Preliminaries: 

• Consider stable LTI system with impulse response h[n] 

• Stable =» e j(a € ROC => 

H{ CO) = Y, h W e ~ jl ° n 

n 

= ^h[n]z~ n \ z = e i< a = H (z)\z= e i<*> (5.35) 

n 

• Recall that: h[n) € M <*=> H* (to) = //(-to) //*(z) = H(z*) 
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Fig. 5.8 Frequency response of a Moving Average system (M = 8). 


Properties: 

The squared magnitude response of a stable LTI system can be expressed in terms of the system function as 
follows: 


|//(m)| 2 = H(z)H*(l/z*)\ z=e jm 

= H(z)H(l/z)\ z= e i a if h[n] € M 

Proof: Using (5.35), the square magnitude response can be expressed as 

|//(co)| 2 = //(«)//* (to) =H(z)H*(z)\ z=e i'» 
Note that when z = e /w , we can write z = 1 /z*. Hence: 

H*(z) = H*(l/z*)\ z =eJ<° 


When h [n] G M, we have 


H*(z)=H(z*)=H(l/z)\ z = e j» 


To complete the proof, substitute (5.38) or (5.39) in (5.37) □. 


(5.36) 


(5.37) 


(5.38) 

(5.39) 


Magnitude square function: 

• C{z) = H{z)H*{\/z*) 
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• Important PZ property: 

- Zk = zero of H(z) =>• Zk and 1 /z* k are zeros of C(z) 

- pk = pole of H(z) => Pk and 1 /p\ are poles of C(z) 

• This is called conjugate reciprocal symmetry (see Figure 5.9). 



Fig. 5.9 Typcial Zero/Pole plot for a magnitude square function C{z). 

• Suppose magnitude response \H(oi) | 2 is known as a function of co, as well as the number of poles and 
zeros of H(z)'- 

- we can always find the PZ diagram of C(z) 

- from there, only a finite number of possibilities for H(z ) 

5.6 All-pass systems 

Definition: 

We say that LTI system ‘M is all-pass (AP) iff its frequency response, H( co), satisfies 

j//(co)| = K , for all co <E (5.40) 

where K is a positive constant. In the sequel, this constant is set to 1. 

Properties: 

Let H(z) be an AP system. 
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• From definition of AP system, 

|Z/(co) | 2 =H(z)H*(l/z*)\ z = e jm = 1, for all to e [-7t,7t] 

This implies 

\H(z)H*( 1/z*) = 1, for allz E C 


• From above relation, it follows that 

= 1 /H*(z) 

Therefore 


(5.41) 

(5.42) 

(5.43) 


z 0 = zero of order I of H(z) 1/z* = pole of order / of H(z) 

p 0 = pole of order 1 of H(z) 1 /p* a = zero of order 1 of H(z) 


Trivial AP system: 

• Consider integer delay system system: //(go) = e ~ 

• This is an all-pass system: |Z/(co) | = 1 


First order AP system: 

From the above PZ considerations, we obtain the following PZ diagram: 



System function: 

Hap(z) = G 

z — a 



1 - az -1 ’ 


where the system gain G has been chosen such that H(z) = 1 at z = 1. 
The frequency response is shown in Figure 5.10. 


(5.44) 
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Fig. 5.10 Frequency response of a first order all-pass system. 


General form of rational AP system: 

• Consider rational system: 


H(z) = 


In order for this system to be AP, we need: 


B(z) _ ,- L LUhz- k 


= z 


A (z) " Lk= 0 a kZ~ 


B{z) = Z~ N A*( 1/z*) 


N 


= E 4z‘ 

k =0 


*„k—N 


• AP system will be causal and stable if, in addition to above: 

- L>0 

- all poles (i.e. zeros of A(z)) inside U.C. 


(5.45) 


(5.46) 


Remark: 

• Consider two LTI systems with freq. resp. H\ (co) and //o ( 0 J). 
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• \H\ (to) | = |tf 2 (co)| for all co e iff H 2 (z) = A (z)H ap (z) 

for some all-pass system H ap (z ) 

5.7 Inverse system 

Definition: 

A system “H is invertible iff for any arbitrary input signals ai and a 2 , we have: 

xi ^ X2 = => lAx\ / tHx 2 (5.47) 


Definition: 

Let 9{ be an invertible system. Its inverse, denoted 9{j, is such that for any input signal a in the domain of 
we have: 

Ui(fKx)=x (5.48) 

Remarks: 

• Invertible means that there is a one-to-one correspondence between the set of possible input signals 
(domain of lA) and the set of corresponding output signals (range). 

• When applied to output signal y = lUx, the inverse system produces the original input a 



Fundamental property: 


Let ‘Jl be an invertible LTI system with system function H ( 7 ). The inverse system ‘A) is also LTI with system 
function 


Hi{z) = 1 /H(z) 


(5.49) 


Remarks: 

• Several ROC are usually possible for Hj(z) (corresponding to causal, mixed or anti-causal impulse 
response hj[n\) 

• Constraint: ROC(.W) 0 ROC( ‘A)) cannot be empty 

• From (5.49), it should be clear that the zeros of H(z) become the poles of Hiiz) and vice versa (with 
the order being preserved). 
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• Thus, causal and stable inverse will exist iff all the zero of Hj(z) are inside the unit circle. 

Proof (the first two parts are optional): 

- Linearity: Let y i = Hx\ and y 2 = Hx 2 . Since H is linear by assumption, we have H{a\x\ + a 2 x 2 ) = 
fl 2 yi + a 2 y 2 . By definition of an inverse system, 

H(a 2 y\ +fl 2 y 2 ) = ci\X\ + a 2 x 2 = a\H(y{) + a 2 Hi(y 2 ) 

which shows that ‘Jl) is linear. 

- Time invariance: Let y = Hx and let denote the shift by k operation. Since H is assumed to be 
time-invariant, H ( T\x) = Dyy. Therefore, HpfiDiy) = Tfx = 1\{Hy) which shows that H} is also 

TI. 

- Eq. (5.49): For any x in the domain of H, we must have HfiHx) = v. Since both H and 'Ll) are LTI 
systems, this implies that 

H I (z)H(z)X(z)=X(z) 

from which (5.49) follows immediately □. 


Example 5.4: 

► Consider FIR system with impulse response (0 < a < 1): 

h[n] = 8[n] — ad[n — 1] 

The corresponding system function 

H(z) = 1 -flz -1 , |z| > 0 

Invoking (5.49), we obtain 


Hfiz) = 

The corresponding PZ diagrams are shown below 
f lm(z) 

H(z) 


1 


1 — az 1 


-o- 


Re(z) 


unit circle 



Note that there are two possible ROC for the inverse system Hfiz)'. 

• ROCi : |z| > a => 

h;[n] = a"u[n] causal & stable 

• ROC 2 : |z| < a => 

/j/[«] = — a n u[—n — 1] anti-causal & unstable 


◄ 
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5.8 Minimum-phase system 

5.8.1 Introduction 

It is of practical importance to know when a causal & stable inverse -H) exists. 


Example 5.5: Inverse system and ROC 

► Consider the situation described by the pole/zero plot in Figure 5.11 (note: llf(z) = 1 / fl(z')). There are 
3 possible ROC for the inverse system ‘Ji)\ 

- ROCi : |z| < 1/2 => anti-causal & unstable 

- ROCi : 1 /2 < |z| < 3/2 => mixed & stable 

- ROCi : |z| > 3/2 => causal & unstable 

A causal and stable inverse does not exist! ◄ 




Fig. 5.11 Example of pole/zero plot of an LTI system and its inverse. 


From the above example, it is clear that: 

poles of Hj(z) inside u.c. 
zeros of H(z) inside u.c. 

These considerations lead to the definition of minimum phase systems, which fulfill the above conditions. 


9{i causal and stable 


Definition: 

A causal LTI system is said to be minimum phase (MP) iff all its poles and zeros are inside the unit circle. 


Remarks 

• Poles inside u.c. =>• h[n] can be chsoen to be causal and stable 

Zeros inside u.c. =>• h/\n] can be chsoen to be causal and stable exists 
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• Minimum phase systems play a very important role in practical applications of digital filters. 


5.8.2 MP-AP decomposition 

Any rational system function can be decomposed as the product of a minimum-phase and an all-pass com¬ 
ponent, as illustrated in Figure 5.12: 

H{z) = H min {z)H ap {z) (5.50) 

MP AP 



Fig. 5.12 Minimum phase/All pass decomposition of a rational system. 


Example 5.6: 

► Consider for instance the transfer function 


(1 1 Z - 1 )( 1 _ 3 Z -1 




1 —z 1 + \z 2 


whose pole/zero plot is shown in Figure 5.11. The zero at z = 3/2 is not inside the unit circle, so that 
H(z) is not minimum-phase. The annoying factor is thus 1 — \z~ x on the numerator of H(z), and it will 
have to be included in the all-pass component. This means that the all-pass component H ap [z ) will have 
a zero at z = 3/2, and thus a pole at z = 2/3. The all-pass component is then 


H ap (z) = G 



l 


l' 


while H min (z ) is chosen so as to have H{z) = H min (z)H up (z): 


H(z) 1 (1 ~\z ‘)(1 — |z 1 
H ap (z ) G l-z _1 + jz _ 


The corresponding pole-zero plots are shown in Figure 5.13. If we further require the all-pass component 
to have unit gain, i.e. H ap (z = 1) = 1, then we must set G = —3/2. ◄ 


5.8.3 Frequency response compensation 

In many applications, we need to remove distortion introduced by a channel on its input via a compensating 
filter (e.g. channel equalization), like illustrated in Figure 5.14. 

Suppose //((o) is not minimum phase; in this case we know that there exists no stable and causal inverse. 
Now, consider the MP-AP decomposition of the transfer function: H( oo) = H mm (to) H a/ ,( to) A possible 
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Fig. 5.13 Pole/Zero plot for a decomposition in Minimum-phase and all-pass parts. 


x\n\ 

Distorting 

y[n] 

Compensating 

x[n ] 

system 

system? 


H(a)) 


H c (co) 



Fig. 5.14 Frequecny response Compensation 


choice for the compensating filter (which is then stable and causal) is the inverse of the minimum phase 
component of //(at)): 

// c (to) = 1///„„■„ (co). (5.51) 

In this case, the compounded frequency response of the cascaded systems in Figure 5.14 is given by 

= H c ((o)H( at) = //^(at), (5.52) 

Accordingly, the magnitude spectrum of the compensator output signal x[n] is identical to that of the input 
signal x[n\, whereas whereas their phases differ by the phase response of the all pass component H ap (z)- 

|X(oo)| = \X (at) | exact magnitude compensation (5.53) 

ZX(b l) = ZX(m) + ZH up (m) => phase distortion 

Thanks to this decomposition, at least the effect of //(oo) on the input signal’s magnitude spectrum can be 
compensated for. 


5.8.4 Properties of MP systems 

Consider a causal MP system with frequency response //„„•„(a>). For any causal system //(oo) with the same 
magnitude response (i.e. |//(oo)| = j// m ,„(co)|): 

(1) group delay of //(oo) > group delay //,„,,, (oo) 
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M M 

(2) ^ h\n] | 2 < ^ for all integer M > 0 

n —0 n—0 


According to (1), the minimum phase system has minimum processing delay among all systems with the 
same magnitude response. Property (2) states that for the MP system, the energy concentration of the 
impulse response h min [n] around n = 0 is maximum (i.e. minimum energy delay). 


5.9 Problems 

Problem 5.1: Pole-zero plots 

For each of the pole-zero plots in figure 5.15, state whether it can correspond to 

1. an allpass system; 

2. a minm um phase system; 

3. a system with real impulse response; 

4. an FIR system; 

5. a Generalized Linear Phase system; 

6. a system with a stable inverse. 

In each case, specify any additional constraint on the ROC in order for the property to hold. 

Problem 5.2: Inversion Moving Average system 

An /V/-point moving average system is defined by the impulse response 

%] = ^(w[n]-n[n-M]). 

1. find the z-transform of h[n\ (use the formula for a sum of a geometric series to simplify your expres¬ 
sion); 

2. Draw a pole-zero plot of H(z)- What is the region of converhence ? 

3. Compute the z- transform for all possible inverses hj[n] of h\n]. Is any of these inverses stable ? What 
is the geenral shape of the correspndiong impulse responses hi[n]l 

4. A modified Moving Average system with forgetting factor is 

h[n] = — a n (u[n] — u[n — M]), 

where a is real, positive and a < 1. Compute G(z) and draw a pole-zero plot. Find a stable and causal 
inverse G/(z) and compute its impulse response g/\n]. 
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Fig. 5.15 Pole-Zero plots pertaining to problem 5.1. 
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Problem 5.3: Implementation of an approximate noncausal inverse 

Let a system be defined by 

i -for 1 

H{z) = - -rr> F > 

1 — az 1 

where \b\ > 1 > |a|. 

1. Is this system stable? Is it causal ? 

2. Find the impulse response h[n\. 

3. Find the ^-transforms Hi(z) of all possible inverses for this system. In each case, specify a region of 
convergence, and whether the inverse is stable and/or caussal. 

4. For each of the above inverses, compute the impulse response and skecth it for a = 0.5, b = 1.5. 

5. Consider the anticausal inverse hj[n\ above. You want to implement a delayed and truncated version 
of this system. Find the number of samples of hi[n\ to keep and the corresponding delay, so as to keep 
99.99% of the energy of the impulse response hj[n]. 


Problem 5.4: Minimum-phase/All-Pass decomposition 

Let a system be defined in the z domain by the following set of poles and zeros: 

• zeros: 2, 0.7e ±77t//8 ; 

• poles: l, 0.3e ±77t ' /l2 . 

Furthermore, H(z) is equal to 1 at z = 1. 

1. Draw a pole-zero plot for this system; 

2. Assume that the system is causal. Is it stable ? 

3. Compute the factorization of H(z) into 

H{z) = H min {z)H ap {z), 

where H in i„(z ) is minmimum-phase , and H ap (z ) is allpass. Draw a pole-zero plot for and 

Hap(z). 


Problem 5.5: Two-sided sequence 

Let a stable system have the transfer function 


H(z) 


1 

(1 - |z — 1 )(1 - \e^! A z~ x ){ 1 - \e~i n ! A z~ l ) 
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1. Draw a pole-zero plot for H(z) and specify its ROC; 

2. compute the inverse z-transform h\n\. 
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Chapter 6 

The discrete Fourier Transform (DFT) 


Introduction: 

The DTFT has proven to be a valuable tool for the theoretical analysis of signals and systems. However, if 
one looks at its definition, i.e.: 


X(co) = 52 x [ n ] e > co e [—7C,7E], (6.1) 


from a computational viewpoint, it becomes clear that it suffers several drawbacks Indeed, its numerical 
evaluation poses the following difficulties: 

• the summation over n is infinite; 

• the variable to is continuous. 

In many situations of interest, it is either not possible, or not necessary, to implement the infinite summation 

Iin (6.1): 

• only the signal samples x[n\ from n = 0 to n = N — 1 are available; 

• the signal is known to be zero outside this range; or 

• the signal is periodic with period N. 

In all these cases, we would like to analyze the frequency content of signal x[n] based only on the finite set 
of samples jc[0] , jc[1] ,... ,x[N — 1]. We would also like a frequency domain representation of these samples 
in which the frequency variable only takes on a finite set of values, say co* for k = 0,1,... ,N — 1, in order to 
better match the way a processor will be able to compute the frequency representation. 


The discrete Fourier transform (DFT) fulfills these needs. It can be seen as an approximation to the DTFT. 
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6.1 The DFT and its inverse 

Definition: 

The Appoint DFT is a transformation that maps DT signal samples {jc[0], ... .x[N — 1]} into a periodic se¬ 
quence X [&], defined by 

N -1 

X[k] = DFT/v{v[n]} = £ x[n\e~ j2Kkn/N , k£ Z (6.2) 

n —0 


Remarks: 

• Only the samples jc[0], ...,x[N — 1] are used in the computation. 

• The /V-point DFT is periodic, with period N: 

X[k + N] =X[k\. (6.3) 

Thus, it is sufficient to specify X[k] for k = 0,1, ...,N — 1 only. 

• The DFT X[k] may be viewed as an approximation to the DTFT X(co) at frequency co^ = 2nk/N. 

• The ”D” in DFT stands for discrete frequency (i.e. C 0 / c ) 

• Other common notations: 

N -1 

X[k] = x[n\e~^ kn where C 0 /i = 2nk/N (6.4) 

n—0 

N—l 

= x[n)Wtf n where Wn = (6.5) 

n —0 


Example 6.1: 

► (a) Consider 

We have 

(b) Let 


N -1 


X[k]= 52 x[n]e 

n =0 


x[n] = a ", 


n = 0, 

jlnkn/N =1) alU . eZ 

n = 0,l,... ,N — 1 
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We have 


a n e ~i 2nkn / N 


N -1 

m = E 

n—O 

= E Pi where p k=ae~ j2nk ^ N 


1-p 

Note the following special cases of interest: 



otherwise. 


a = e 


a = 1 = 

=* m = 

J2nl/N _ 

=* m = 


N if k = 0, 

[0 ifk = 1. 

' N if k = l modulo N, 
0 otherwise. 


Inverse DFT (IDFT): 

The /V-point IDFT of the samples X[0],... ,X[N — 1] is defined as the periodic sequence 

i N -1 

x[n\ = IDFT;v{X[yt]} = - E X[k]e j2Kkn/N , ne Z (6.6) 

N k =o 


Remarks: 

• In general, x[n] ^ x[n] for all n G Z (more on this later). 

• Only the samples X[0], ...,X[N — 1] are used in the computation. 

• The /V-point IDFT is periodic, with period N: 

x[n+N] = x\n] (6.7) 

• Other common notations: 

i N -1 

x[n] = — E X[k\e-’ (0kn where co* = 27C k/N (6.8) 

N k =o 
i n- t 

= ~Y,X[k]W^ kn where W N = e~ j2n ' N (6.9) 

N k =o 


IDFT Theorem: 

If X [k] is the iV-point DFT of {x [0],..., x [77 — 1 ]}, then 

x[n] =x[n] =IDFT A ,{X[fc]}, n = 0, 1 (6.10) 
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Remarks: 

• The theorem states that x[n] = x[n] for n = 0. I...../V — I only. Nothing is said about sample values of 
x[n] outside this range. 

• In general, it is not true that x[n] = x[n] for all n E Z: the IDFT x\n is periodic with period N, whereas 
no such requirement is imposed on the original signal x[n\. 

• Therefore, the values of x[n] for n < 0 and for n > N cannot in general be recovered from the DFT 
samples X [k]. This is understandable since these sample values are not used when computing X [k]. 

• However, there are two important special cases when the complete signal x[n] can be recovered from 

the DFT samples X[k] (k = 0,1 — 1): 

- x[n] is periodic with period N 

- x[n] is known to be zero for n < 0 and for n>N 


Proof of Theorem: 

• First note that: 

l N y e -p.nkn/N = { 1 if n = 0, ±N, ±2N ,... 

N , n 1 0 otherwise 

k—0 v 

Indeed, the above sum is a sum of terms of a geometric series. If n = IN, / E Z, then all the N terms 
are equal to one, so that the sum is N. Otherwise, the sum is equal to 


l _ e -fixkn 
1 _ e -p-™lN ’ 


whose numerator is equal to 0. 


• Starting from the IDFT definition: 


x\n\ 


N -1 


= 


Jlnkn /N 


k =0 
N -1 (N -1 


N 


L i L x ^ e 


—jlnkl/N l e pnkn/N 


k =0 I 1=0 


N -1 

= £ 4 '] 

1=0 


N -1 


,-P%k(l-n)/N 


N 


k =o 


( 6 . 12 ) 


For the special case when n E {0,1, ...,fV — 1}, we have —N < l — n < N. Thus, according to (6.11), 
the bracketed term is zero unless / = n. Therefore: 


x[n] =x[n], n = 0,1,... ,N — 1 □ 


(6.13) 
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6.2 Relationship between the DFT and the DTFT 


Introduction 


The DFT may be viewed as a finite approximation to the DTFT: 



at frequency co = to* = 2 nk/N As pointed out earlier, in general, an arbitrary signal x[n], n £ Z, cannot be 
recovered entirely from its /V-point DFT X\k].k £ {(). ....N— I }. Thus, it should not be possible to recover 
the DTFT exactly from the DFT. 

Flowever, in the following two special cases: 

- finite length signals 

- /V-pcriodic signals, 

x[n\ can be completely recovered from its /V-point DFT In these two cases, the DFT is not merely an 
approximation to the DTFT: the DTFT can be evaluated exactly, at any given frequency co £ [—if the 
/V-point DFT X[k] is known. 

6.2.1 Finite length signals 

Assumption: Suppose x[n] = 0 tor n < 0 and for n > N. 

Inverse DFT: In this case, x[n] can be recovered entirely from its /V-point DFT X[k], k = 0. 1..... /V — 1 Let 
x [n] denote the IDFT of X [k]: 


x[n] = IDFT{X[k]} = - £ X[k\e j2nkn/N , n £ Z. 


(6.14) 


For n = 0,1, ...,1V — 1, the IDFT theorem yields: x[n] = x[n). For n < 0 and for n > N, we have x[n] = 0 by 
assumption. Therefore: 



(6.15) 


Relationship between DFT and DTFT: Since the DTFT X((o) is a function of the samples x[n], it should 
be clear that in this case, it is possible to completely reconstruct the DTFT X (to) from the IV-point DFT X\k], 

First consider the frequency co* = 2nk/N: 


X(<o k ) = £ 4n]e~ j(akn 


= £ x[n}e- Jb>tn = X[k] 


(6.16) 


The general case, i.e. a) arbitrary, is handled by the following theorem. 
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Theorem: Let X(co) and X[k] respectively denote the DTFT and /V-point DFT of signal x[n]. If x[n\ = 0 
for n < 0 and for n > 0, then: 

N -1 

X(to)= £x[*]P(a>-a>*) (6.17) 

k =0 

where 

P(to) = jF"'" (6.18) 

ly n =0 


Remark: This theorem provides a kind of interpolation formula for evaluating X (co) in between adjacent 
values ofX(©*) = X [k]. 

Proof: 


X(co) 


N -1 

L x[n\e- jmn 

n —0 



N -1 

^ X[k]P(co- co*) □ 

k=0 


Properties of P(oo): 

• Periodicity: P(oo + 27t) = P(co) 

• If 0) = 2nl (/ integer), then e~-’ m = e~^ 2nln = 1 so that P(oo) = 1. 

• If co / 27t/, then 


P(co) 


1 l-e~ j(oN 
N 1 - eJ' a 

1 _-,- m fiy-i )/2 sin(ta/V/2) 
/V sin(co/2) 


• Note that at frequency Ofy. = 2nk/N: 


P(cda-) 


1 k = 0, 

0 P = 1, ...,7V— 1. 


so that (6.17) is consistent with (6.16). 

• Figure 6.1 shows the magnitude of P(co) (for N = 8). 


(6.19) 


( 6 . 20 ) 


( 6 . 21 ) 
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|P(m)l 



Fig. 6.1 Magnitude spectrum of the frequency interpolation function P(<») for N = 8. 


Remarks: More generally, suppose that x[n\ = 0 for n < 0 and for n > L. As long as the DFT size A is 
larger than or equal to L, i.e. A >L, the results of this section apply. In particular: 

• One can reconstruct x[n] entirely from the DFT samples X[k] 

• Also, from the theorem: X[k\ = X ( 0 J/, : ) at (Ok = 2nk/N. 

Increasing the value of N beyond the minimum required value, i.e. N = L, is called zero-padding : 

• {x[0],...,x[L-l]} => {x[0],...,x[L— 1],0,...,0} 

• The DFT points obtained give a nicer graph of the underlying DTFT because Aa>k = 2n/N is smaller. 

• Flowever, no new information about the original signal is introduced by increasing N in this way. 


6.2.2 Periodic signals 

Assumption: Suppose x[n] is /V-pcriodic, i.e. x\n + A] = x\n\. 

Inverse DFT: In this case, x[n] can be recovered entirely from its A-point DFT X[k], k = 0,1,..., A — 1. 
Let x[n] denote the IDFT of X[k], as defined in (6.14). For n = 0,1,...,A — 1, the IDFT theorem yields: 
x[n] = x[n]. Since both x[n ] and x\n\ are known to be A-periodic, it follows that x[n] = x[n] must also be true 
for n < 0 and for n > A. Therefore 

x[n] = Jc[n], Mn G Z (6.22) 
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Relationship between DFT and DTFT: Since the /V-pcriodic signal x[n] can be recovered completely 
from its /V-point DFT X[k], it should also be possible in theory to reconstruct the DTFTX(co) from X\k}. 

The situation is more complicated here because the DTFT of a periodic signal contains infinite impulses. 
The desired relationship is provided by the following theorem. 


Theorem: Let X(co) and X\k] respectively denote the DTFT and Appoint DFT of ^-periodic signal x[n. 
Then: 

2k °° 

X(co) = — £ X[*]8 0 (a>-a>*) (6.23) 

k——oo 

where 8 a (co) denotes an analog delta function centered at to = 0. 


Remarks: According to the Theorem, one can recover the DTFT X((o) from the /V-point DFT X[k}. X(co) 
is made up of a periodic train of infinite impulses in the to domain (i.e. line spectrum). The amplitude of the 
impulse at frequency co k = 2 tc k/N is given by 2nX[k]/N 

Proof of Theorem (optional): 

oo 

X(co) = £ x[n}e- jmn 

n——oo 

OO ( 1 N~\ ^ 

= E ;[«<* '- 1 * 

«=~°° [ /v k=0 J 

= lE X W i e^ n e-j™ 

^ k= 0 n=—“ 

i tV-1 

= - £x[£]DTFT{e /Q) *"} 

N k=o 
1 tv-i 

= - £x[£]2tc £ 8 a (co- (a k -2nr) 

M k =0 r=-oo 

r\ oo N —1 OtT 

= T 7 £ £*[£]8 a (co -—(k + rN)) (6.24) 

iV r=-oo k=0 

Realizing that X[k\ is periodic with period N, and that Y.7—^f(k + r N) i s equivalent to Y7k=-oof(k) 
we finally have: 


?Tt “ N ~ X ?lt 

*(») = f E £x[i + -W]8„(<0--A*: + ,W)) 

■'V r= _oo k=0 M 

= %im □ 

k =—oo 

Remarks on the Discrete Fourier series: In the special case when x[n] is A-periodic, i.e. x[n+N] = x[n], 
the DFT admits a Fourier series interpretation. Indeed, the IDFT provides an expansion of x[n] as a sum of 
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harmonically related complex exponential signals e 7 ® 4 ": 

i N-l 

x[n] = - £ X[k]e j( ° kn , neZ (6.25) 

N k =0 

In some textbooks (e.g. [10]), this expansion is called discrete Fourier series (DFS). The DFS coefficients 
X[k\ are identical to the DFT: 

N -1 

X[k] = £ x[n]e~ j(0kn , neZ (6.26) 

n— 0 

In these notes, we tie at the DFS as a special case of the DFT, corresponding to the situation when x[n] is 
/V-pcriodic. 


6.3 Signal reconstruction via DTFT sampling 
Introduction: 

Let X(co) be the DTFT of signal x[n\, n G Z, that is: 


X(co)= £ x[n]e~ j(an (OGl. 

n——oo 

Consider the sampled values of X(oo) at uniformly spaced frequencies co k = 2nk/N for k = 0..... A 7 — 1. 
Suppose we compute the IDFT of the samples X(cor): 


x[n] = IDFT{X(co,)} = 1' X(®k)e j(0kn (6.27) 

™ k= 0 

What is the relationship between the original signal x[n] and the reconstructed sequence x [«]? 

Note that x[n] is N-periodic, while x\n] may not be. Even for n = 0 ,...,N — 1, there is no reason for x\n] to 
be equal to x[n\. Indeed, the IDFT theorem does not apply since X(( 0 / { ) / DFT f v{x[/;]}. 

The answer to the above question turns out to be very important when using DFT to compute linear convo¬ 
lution... 


Theorem 


oo 

x[n] = IDFT{X(co a .)} = Yj x \ n ~ rN \ 

r——oo 


(6.28) 
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Proof: 


x\n\ = 


N -1 


= £ X(w k )e 


J(O k n 


k =0 
At-1 


= jv 11 I x ® e 


l £ j«> k n 


k =0 l/= 


JV-1 


= I i jy I * 


j2n(n—l)k/N 


l——o 


(6.29) 




From (6.11), we note that the bracketed quantity in (6.29) is equal to 1 if n — l = rN, i.e. / = n — rN (r 
integer) and is equal to 0 otherwise. Therefore 


x\n 


= x i n ~ r N) □ 


(6.30) 


Interpretation 

x[n] is an infinite sum of the sequences x[n — rN], r G Z. Each of these sequences x\n — rN] is a shifted 
version of x[n] by an integer multiple of N. Depending on whether or not these shifted sequences overlap, 
we distinguish two important cases. 


• Time limited signal: Suppose x[n] = 0 for n < 0 and for n > N. Then, there is no temporal overlap of 
the sequences x[n — rN]. We can recover x[n] exactly from one period of x[n ]: 

jx[n] n = 0, l,...,N — 1, 

x[n\ = < (6.31) 

10 otherwise. 

This is consistent with the results of Section 6.2.1. Figure 6.2 shows an illustration of this reconstruc¬ 
tion by IDFT. 

• Non time-limited signal: Suppose that x[n] f 0 for some n < 0 or n > N. Then, the sequences x[n — rN] 
for different values of r will overlap in the time-domain. In this case, it is not true that x[n] = x[n] for 
allO < n < N — 1. We refer to this phenomenon as temporal aliasing, as illustrated in Figure 6.3. 


6.4 Properties of the DFT 

Notations: 

In this section, we assume that the signals x[n] and y[n] are defined over 0 < n < N — 1. Unless explicitly 
stated, we make no special assumption about the signal values outside this range. We denote the /V-point 
DFT of x[n] and y[n] by X[k) and Y[k): 

r 1 DFT/V r 1 1 

x[n\ <—» X[k\ 

y[n] ^ Y[k] 

We view X[k] and Y\k] as /V-periodic sequences, defined for all k G Z 
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original signal DTFT 




inverse DFT sampled DTFT 




Fig. 6.2 Illustration of reconstruction of a signal from samples of its DTFT. Top left: the 
original signal x[n\ is time limited. Top right, the original DTFT X(co), from which 15 samples 
are taken. Bottom right: the equivalent impulse spectrum corresponds by IDFT to a 15-periodic 
sequence x{n] shown on the bottom left. Since the original sequence is zero outside 0 < n < 14, 
there is no overlap between replicas of the original signal in time. 
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ofiginal signal DTFT 



Fig. 6.3 Illustration of reconstruction of a signal from samples of its DTFT. Top left: the 
original signal jc[/i] is time limited. Top right, the original DTFT X (co), from which 6 samples 
are taken. Bottom right: the equivalent impulse spectrum corresponds by IDFT to a 6-periodic 
sequence x[n] shown on the bottom left. Since the original sequence is not zero outside 0 < 
n < 5, there is overlap between replicas of the original signal in time, and thus aliasing. 
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Modulo N operation: 

Any integer n e Z can be expressed uniquely as n = k + rN where 
k € {0,1, ...,N — 1} and r € Z. We define 


(n)N = n modulo N = k. 


(6.32) 


For example: (ll)s = 3, (—1)6 = 5. 

Note that the sequence defined by *[(«)#] is an /V-pcriodic sequence made of repetitions of the values of x[n] 
between 0 and N — 1, i.e. 

oo 

x[(n) N \ = Y. n ~ rN ] (6- 33 ) 

r ——oo 

where ^[n] =x[n](n[n] —n[n —A]). 


6.4.1 Time reversal and complex conjugation 

Circular time reversal: Given a sequence x[n], 0 < n < N — 1, we define its circular reversal (CR) as 
follows: 

CR{r[n]} =r[(-n)j\r], 0<n<N— 1 (6.34) 


Example 6.2: 

► Let x[n\ = 6 — « for« = 0,1,..,5: 


n 

0 

1 

2 

3 

4 

5 

x[n\ 

6 

5 

4 

3 

2 

1 


Then, for N = 6, we have CR{x[n]} = x[(— n)(]: 


n 

0 

1 

2 

3 

4 

5 

(-«)e 

0 

5 

4 

3 

2 

1 

*[(-*)e] 

6 

1 

2 

3 

4 

5 


This is illustrated Figure 6.4. 



CR 




< 


Fig. 6.4 Illustration of circular time reversal 
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Interpretation: Circular reversal can be seen as an operation on the set of signal samples jt[0],... ,x\N — 1]: 

• jc[ 0] is left unchanged, whereas 

• for k = 1 to N — 1, samples x\k\ and x[N — k] arc exchanged. 

One can also see the circular reversal as an operation consisting in: 

• periodizing the samples of x[n\, 0 < n < N — 1 with period N: 

• time-reversing the periodized sequence; 

• Keeping only the samples between 0 and N —l. 

Figure 6.5 gives a graphical illustration of this process. 


x[n] 



Fig. 6.5 Graphical illustration of circular reversal, with N = 15. 
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Property: 


x[(-n) N \ X[-k\ 

x*[n\ 2^ X*[-k] 
x*[{~n) N ] S X*[k) 


(6.35) 

(6.36) 

(6.37) 


Remarks: 

• Since X[k\ is fV-periodic, we note that X[—k] = X[(— k)^]. For example, if N = 6, then X [—1] = X[5] 
in (6.35). 

• Property (6.36) leads to the following conclusion for real-valued signals: 

x[n\=x*[n\^X[k\=X*[-k\ (6.38) 

or equivalently, iff |X[fc]| = |X[— k] \ and XX[k] = —XX[—k\. 

• Thus, for real-valued signals: 

- X[0] is real; 

- if TV is even: X[N/ 2] is real; 

- X[N-k] =X*[k] for 1 <k<N- 1. 

Figure 6.6 illustrates this with a graphical example. 


6.4.2 Duality 
Property: 

X[n\ S A6c[(-£)jv] 


Proof: First note that since (— k)^ = — k + rN, for some integer r, we have 

e -j2nkn/N _ e j2nn(-k) N /N 

Using this identity and recalling the definition of the IDFT: 


N ^X[n\e~ j2mklN 

n —0 


^ X[n\e j2m( ~ k)N/N 

n —0 

N x IDFT{X[n]} at index value (—k)^ 
Nx[(-k) N ] □ 


(6.39) 


(6.40) 
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x[n] 


0 

4 

( 

1 

i 

.f 
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Fig. 6.6 Illustration of the symmetry properties of the DFT of a real-values sequence (N = 
10 ). 
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6.4.3 Linearity 
Property: 

ax[n] + by[n\ <—5 aX[k\+bY[k\ (6.41) 


6.4.4 Even and odd decomposition 

Conjugate symmetric components: The even and odd conjugate symmetric components of finite se¬ 
quence x[n\ , 0 < n < N — 1, are defined as: 

XeA n \ — ^(*[n]+**[(-n)tvD (6-42) 

XoA n ] - *(*[«]-**[(-«)tv]) (6.43) 

In the case of a IV-periodic sequence, defined for n e Z, the modulo N operation can be omitted. Accordingly, 
we define 

X e [k] 4 l -(x[k\+X*[-k\) (6.44) 

Xo[k] = l -(X[k]-X*[-k]) (6.45) 

Note that, e.g. jc e ^v[0] is real, whereas x e ^[N — n] = x e ^[n], n = 1..... A — I. 


Property: 


Re{x[n]} 

DFTa) 

< - > 

X e [k\ 

(6.46) 

j\m{x[n}} 

dft n 

< - > 

Xo[k) 

(6-47) 

x e ,N[n\ 

dft n 
< - > 

Re{X[k}} 

(6.48) 

x 0 ,N[n] 

DFTa, 

< -» 

j\m{X[k]} 

(6.49) 


6.4.5 Circular shift 

Definition: Given a sequence x[n] defined over the interval 0 < n < N — 1, we define its circular shift by k 
samples as follows: 

CS k{x[n]} = x[(n —k)^}, 0<n<N—l (6.50) 


Example 6.3: 

► Let x[n] = 6 — n for n = 0,1, ..,5. 


n 

0 

1 

2 

3 

4 

5 

x\n] 

6 

5 

4 

3 

2 

1 
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For A = 6, we have CSj.{x[n]} = x[(n — k)f\ In particular, for k = 2\ 


n 

0 

1 

2 

3 

4 

5 

(n- 2) 6 

4 

5 

0 

1 

2 

3 

x[{n- 2) 6 ] 

2 

1 

6 

5 

4 

3 


This is illustrated Figure 6.7. ◄ 





Fig. 6.7 Illustration of circular time reversal 


Interpretation: Circular shift can be seen as an operation on the set of signal samples x[n] (n G {0. N — 

1}) in which: 

• signal samples x[n\ are shifted as in a conventional shift; 

• any signal sample leaving the interval 0 <n <N — l from one end reenters by the other end. 
Alternatively, circular shift may be interpreted as follows: 

• periodizing the samples of x[n] ,0<n<N—l with period A; 

• delaying the periodized sequence by k samples; 

• keeping only the samples between 0 and N — 1. 

Figure 6.8 gives a graphical illustration of this process. It is clearly seen from this figure that a circular shift 
by k samples amounts to taking the last k samples in the window of time from 0 to N — 1, and placing these 
at the beginning of the window, shifting the rest of the samples to the right. 

Circular Shift Property: 

x[{n-m) N ] 2^ e~ j2mnk/N X[k\ 

Proof: To simplify the notations, introduce co^ = 2nk/N. 

N -1 

DFT;v{.*[(« - m ) N ]} = Y, x i( n - m) N ]e~ ja3kn 

n—0 

N—l—m 

= e~ j(0km £ *[(») N]e~ jakn 
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x[n] 



Fig. 6.8 Graphical illustration of circular shift, with N = 15 and k = 2. 
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Observe that the expression to the right of is /V-periodic in the variable n. Thus, we may replace 

that summation by without affecting the result (in both cases, we sum over one complete period) 

DFT w {x[(n-m) w ]} = e~^ m x[(n) N ]e~^ n 

n —0 
N—l 

= e~ j(0km Y x[n\e~ j(akn = e~ ja>k "'X[k] □ 

n —0 

Frequency shift Property: 

e j2nmn/N x [„] ^ 


Remark: Since the DFT X[k] is already periodic, the modulo N operation is not needed here, that is: 
X[(k — m)v] =X[k — m] 


6.4.6 Circular convolution 

Definition: Let x[n] and y[n] be two sequences defined over the interval 0 < n < N — 1. The A^-point 
circular convolution of x\n] and y\n\ is given by 

At—1 

x[n]®y[n\ = Y x [m]y[(n — m)x/\, 0<n<N—l (6.51) 

m —0 

Remarks: This is conceptually similar to the linear convolution of Chapter 2 (i.e. £“ =00 = x[m]y\n — m\ ) 
except for two important differences: 

• the sum is finite, running from 0 to N — I 

• circular shift is used instead of a linear shift. 

The use of (n — m )n ensures that the argument of y[] remains in the range 0,1, 1. 

Circular Convolution Property: 

x[n]®y[n\ S X[k]Y[k\ (6.52) 

Multiplication Property: 

jc[«]y[«] 2^5 —X[k\®Y[k] (6.53) 

6.4.7 Other properties 
Plancherel’s relation: 

(6-54) 

n =o iy k =o 
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Parseval’s relation: 



(6.55) 


Remarks: 


• Define signal vectors 


X = [jc[0],...,jc[jv-i]], y = [y[o],...,y[Y-1]] 


Then, the LHS summation in (6.54) may be interpreted as the inner product between signal vectors x 
and y in vector space C^. Similarly, up to scaling factor 1 /N, the RHS of (6.54)may be interpreted as 
the inner product between the DFT vectors 


X = [Y[0],. • • ,Y[N — 1]], Y = [T[0],... ,y[tV— 1]] 


Identity (6.54) is therefore equivalent to the statement that the DFT operation preserves the inner- 
product between time-domain vectors. 

• Eq. (6.55) is a special case of (6.54) with y[n] = x[n]. It allows the computation of the energy of the 
signal samples x[n\ (n = 0 ,...,N — 1) directly from the DFT samples X\k\. 

6.5 Relation between linear and circular convolutions 

Introduction: 

The circular convolution © admits a fast (i.e. very low complexity) realization via the so-called fast Fourier 
transform (FFT) algorithms. If the linear and circular convolution were equivalent, it would be possible to 
evaluate the linear convolution efficiently as well via the FFT. 

The problem, then, is the following: under what conditions are the linear and circular convolutions equiva¬ 


lent? 


In this section, we investigate the general relationship between the linear and circular convolutions. In 
particular, we show that both types of convolution are equivalent if 

- the signals of interest have finite length, 

- and the DFT size is sufficiently large. 

Linear convolution: 

Time domain expression: 


yi [n] = x\ [n] * xi [n] = ^ xi [k\x 2 [n — k], n G Z 


(6.56) 


k——oo 


Frequency domain representation via DTFT: 


7/(co) = X\ (co)Y 2 ((o), to e [0,2 ji] 


(6.57) 
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Circular convolution: 

Time domain expression: 

JV—1 

y c [n\= x\[n]®X 2 [n\= ^ xi[k\x 2 [{n —k)^\, 0 <n<N —\ 

k =0 

Frequency domain representation via /V-point DFT: 

Y c [k\=X l [k]X 2 [k\, k € {0,1} 


(6.58) 


(6.59) 


Observation: 

We say that the circular convolution (6.58) and the linear convolution (6.56) are equivalent if 

r i _ \yM ifO</i<iV 
yi U ~ (o otherwise 

Clearly, this can only be true in general if the signals x\ [n] and xi \n\ both have finite length. 


(6.60) 


Finite length assumption: 

We say that x[n] is time limited (TL) to 0 < n < N iff x[n ] = 0 for n < 0 and for n > N. 

Suppose that jq [n] and X 2 \n\ are TL to 0 < n < N\ and 0 < n < W, respectively. Then, the linear convolution 
yi[n] in (6.56) is TL to 0 < n < N3, where N3 = N\ +N2 — 1. 


Example 6.4: 

► Consider the TL signals 

xi[n) = {1,1,1,1} x 2 [n) = {1,1/2,1/2} 
where N\ = 3 and N 2 = 4. A simple calculation yields 

yA n \ = {1, i- 5 ,2,2,1, - 5 } 

with N 3 = N\ +N 2 — 1 = 6. This is illustrated in figure 6.9. ◄ 


Condition for equivalence: 

Suppose that x\ [n] and x 2 [n] are TL to 0 < n < N\ and 0 < n < /V 2 . 

Based on our previous discussion, a necessary condition for equivalence of yi[n] and y c [n] is that N > 
Ni +N 2 — 1. We show below that this is a sufficient condition. 
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A x\k] 


o 3 


A x 2 [-k] 


o x 2 [£] 


o k 



> y,[ 0] = i y ,W = o,«<o 


A x 2 [l -k] 

>=> y,[ l] = l.5 

o i k 


O 

O 



o 


>’,[5] = .5 y, [/?] = 0,n>6 


Fig. 6.9 Illustration of the sequences used in example 6.4. Top: xi [n] andx 2 [n]; bottom: yi[n\, 
their linear convolution. 
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Linear convolution: 

oo 

yi[n] = Yj X\[k]xi[n-k] 

k——oo 

n 

= Y x i[k]x2[n — k\, 0 <n<N—l (6.61) 

k =o 

Circular convolution: 

N-l 

y c [n\ = Y x i[k]x2[(n-k) N ] 
k =o 

n N-l 

= Y x i\k\ x i[n — k] + Y x\[k\x 2 [N + n — k\ (6.62) 

k —0 k—n -\-1 


If N > N 2 + A/ — 1, it can be verified that the product x\ \k\x 2 \N + n — k\ in the second summation on the 
RHS of (6.62) is always zero. Under this condition, it thus follows that> 7 [n] = y c [n] for 0 < n < N — 1. 

Conclusion: the linear and circular convolution are equivalent iff 


N>N l +N 2 -l 


(6.63) 


Example 6.5: 

► Consider the TL signals 

xi [n] = {1,1,1,1} x 2 [n) = {1,1/2,1/2} 

where A) = 3 and AA = 4. First consider the circular convolution y c = x\ ®x 2 with ;V = /V| -|- A^t 1 =6, 
as illustrated in figure 6.10. 

The result of this computation is 

*[*] = {!, 1 . 5 , 2 , 2 , 1 ,. 5 } 

which is identical to the result of the linear convolution in example 6.4. 

Now consider the circular convolution y c = x\ with A' = 4, as illustrated in figure 6.11. 

The result of this computation is 

yc[n] = {2,2,2,2} 

◄ 


Remarks: 

When IV > Ni + AA — L the CTR operation has no effect on the convolution. 

On the contrary, when N = N[ + N 2 — l, the CTR will affect the result of the convolution due to overlap 
between xi [£] and x 2 [N + n — k]. 
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A x t [k] 


A x 2 [k\ 


1 


0 5 k 

A x 2 [(-k) 6 ] = CRx l [k] 



0 s k 

4 XiV}-*)(,] 


0 1 


5 k 



j> X[0] = 1 


=^> ^[ 1 ] = 1.5 

o 

o 

o 


y e [5] = .5 


Fig. 6.10 Illustration of the sequences used in example 6.5. Top: x\[n\ and X 2 [«]; bottom: 
y c \n\, their circular convolution with N = 6. 
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A *il k] 


o 3 k 

A x 2 [(-k ) 4 ] 

• • 

0 3 k 

A x 2 [(\-k) A ] 


0 1 3 k 



A 



-► 


A: 


7 C [0] = 2 


=> ^[!] = 2 

O 

o 

o 


^[3] = 2 


Fig. 6.11 Illustration of the sequences used in example 6.5. Top: x\ [n] and xj\n]\ bottom: 
y c [n], their circular convolution with N = 4. 
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Relationship between y c [n] and >’/[«]: 

Assume that N > max{Ai, A 2 }. Then the DFT of each of the two sequences x x [n] and x 2 [n\ are samples of 
the corresponding DTFT: 

N >N\ =► X x [k\ = Xi (co*), co* = 27 ik/N 
N>N 2 =► X 2 [k]=X 2 ( co*) 

The DFT of the circular convolution is just the product of DFT: 

Y c [k\ = X x [k\X 2 [k] 

= X 1 (co*)X 2 (co*) = T / (co*) (6.64) 

This shows that Y c [k ] is also made of uniformly spaced samples of X/(co), the DTFT of the linear convolution 
yi[n\. Thus, the circular convolution y c \n] can be computed as the X-point IDFT of these frequency samples: 

Vc[n] = IDFT;v{F c [£]} = IDFT w {7/(co*)} 

Making use of (6.28), we have 

00 

y c [ n ] = 52 yi [n - rN] , 0 < n < N - 1, (6.65) 

r——oo 

so that the circular convolution is obtained by 

- an A-periodic repetition of the linear convolution y/ \n\ 

- and a windowing to limit the nonzero samples to the instants 0 to N — 1. 

To get y c [n] = yi[n\ for 0 < 77 < N — 1, we must avoid temporal aliasing. We need: lenghtofDFT > 
length of yi[n], i.e. 

N>N 3 =N x +N 2 -l (6.66) 

which is consistent with our earlier development. The following examples illustrate the above concepts: 


Example 6.6: 

► The application of (6.65) for N < N\ + N 2 — 1 is illustrated in Figure 6.12. The result of a linear convo¬ 
lution }'[ [n] is illustrated (top) together with two shifted versions by +N and —N samples, respectively. 
These signals are added together to yield the circular convolution y c [n] (bottom) according to (6.65). In 
this case, N = 11 and N 3 = N x + N 2 — l = 15, so that there is time aliasing. Note that out of the N = 11 
samples in the circular convolution y c [n], i.e. within the interval 0 <n< 10, only the first A 3 — A samples 
are affected by aliasing (as shown by the white dots), whereas the other samples are common to yi [n] 
and y c \n]. ◄ 


Example 6.7: 

► To convince ourselves of the validity of (6.65), consider again the sequences xi [n] = {1,1,1,1} and 
x 2 [n\ = {I, -5, .5} where Ai = 3 and N 2 = 4. From Example 6.4, we know that 

= {1,1.5,2,2,1,.5} 
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y : [n] 



o'— 1 —— 1 ————— L - LU — 1 ————— LU — 1 ———————— L 

-10 -5 0 5 10 15 20 


Fig. 6.12 Circular convolution seen as linear convolution plus time aliasing: 
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while from Example 6.5, the circular convolution for IV = 4 yields 

yc[n] = { 2,2,2,2} 

The application of (6.65) is illustrated in Figure 6.13. It can be seen that the sample values of the circular 



-4 0 1 2 3 4 8 


n 


3 ; cW = X^[" _4r ] 

\ 

~2~ 2 2 2 

-1-!-i-t-1-i-1-t-1-t-1-1-1-1-!-1-► 

0 12 3 n 

Fig. 6.13 Illustration of the circular convolution as a time-aliased verion of the linear convo¬ 
lution (see examepl 6.7). 

convolution y c [n], as computed from (6.65), are identical to those obtained in Example 6.5. ◄ 


6.5.1 Linear convolution via DFT 

We can summarize the way one can compute a linear convolution via DFT by the following steps: 


• Suppose that x\ [n] is TL to 0 <n <N\ and X 2 \n\ is TL to 0 < n <Nj 

• Select DFT size (V > Ni + N 2 — 1 (usually, N = 2 K ). 

• For i = 1,2, compute 

Xi [k] = DFTv{,r; [n ]}, 0 < k < N - 1 


• Compute: 


x \ [n] *X 2 [«] 


IDFT W {Xi [k\X 2 [k} \ 0 < n < N - 1 
0 otherwise 


(6.67) 


( 6 . 68 ) 


Remarks: In practice, the DFT is computed via an FFT algorithm (more on this later). For large N, 
say N > 30, the computation of a linear convolution via FFT is more efficient than direct time-domain 
realization. The accompanying disadvantage is that it involves a processing delay, because the data must be 
buffered. 
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Example 6.8: 

► Consider the sequences 

x\ [n] = {1,1,1,1} x 2 [n] = {I,.5,.5} 

where /V| =3 and N 2 = 4. The DFT size must satisfy N > /V| -f N 2 — 1 = 6 for the circular and linear 
convolutions to be equivalent. 

Let us first consider the case N = 6. The 6-point DFT of the zero-padded sequences x\ [n] and x 2 [n\ are: 

Xi[k] = DFT 6 {1,1,1,1,0,0} 

= {4,-1.7321;, 1,01, +1.7321;} 

Xi[k\ = DFT 6 {1,0.5,0.5,0,0,0} 

= { 2 , 1 - 0.866 ;, 1 / 2 , 1 , 1 / 2,1 + 0 . 866 ;} 


The product of the DFT yields 


Y c [k] = { 8 ,-1.5 -1.7321;, 0.5,0,0.5,-1.5- 1.7321;} 


Taking the 6 -point IDFT, we obtain: 


y c [n] = IDFT 6 {F[A:]} 

= {1,15,2,2,1, .5} 

which is identical to yi [n] previously computed in Example 6.4. The student is invited to try this approach 
in the case N = 4. 


6.6 Problems 

Problem 6.1: DFT of Periodic Sequence 

Let x[n] be aperiodic sequence with period N. Let X\k\ denote its Appoint DFT. Let Y\k] represent its 2N- 
point DFT. 

Find the relationship between X [/:] and Y [k]. 

Problem 6.2: DFT of a complex exponential 

Find the /V-point DFT of x[n\ = e i ln P n l N , 
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Chapter 7 

Digital processing of analog signals 


As a motivation for Discrete-Time Signal Processing, we have stated in the introduction of this course that 
DTSP is a convenient and powerful approach for the processing of real-world analog signals. In this chapter, 
we will further study the theoretical and practical conditions that enable digital processing of analog signals. 


Structural overview: 



Fig. 7.1 The three main stages of digital processing of analog signals 
Digital processing of analog signals consists of three main components, as illustrated in Figure 7.1: 

• The A/D converter transforms the analog input signal x a {t) into a digital signal xj[n]. This is done in 
two steps: 

- uniform sampling 

- quantization 

• The digital system performs the desired operations on the digital signal xj [n] and produces a corre¬ 
sponding output y c i [n] also in digital form. 

• The D/A converter transforms the digital output y ( i[n into an analog signal y a (t) suitable for interfac¬ 
ing with the outside world. 
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In the rest of this chapter, we will study the details of each of these building blocks. 


7.1 Uniform sampling and the sampling theorem 

Uniform (or periodic) sampling refers to the process in which an analog, or continuous-time (CT), signal 
x a {t), t e M, is mapped into a discrete-time signal x\n], n G Z, according to the following rule: 

x a (t) —>x[n] = x a (nT s ), lie Z (7.1) 

where T s is called the sampling period. Figure 7.2 gives a graphic illustration of the process. 




samples 


Fig. 7.2 Illustration of Uniform Sampling: top, the analog signal, where the sampling points 
t = nT Sl n £ Z are shown by arrows; bottom: the corresponding discrete-time sequence. Notice 
the difference in scale betwwen the x-axis of the two plots. 


Based on the sampling period T s , the Sampling Frequency can be defined as 

F s = \/T s , or (7.2) 

a, = 2nF s = 2n/T s , (7.3) 

We shall also refer to uniform sampling as ideal analog-to-discrete-time (A/DT) conversion. We say ideal 
because the amplitude of x[n] is not quantized, i.e. it may take any possible values within the set M. 

Ideal A/DT conversion is represented in block diagram form as follows: 

In practice, uniform sampling is implemented with analog-to-digital (A/D) converters. These are non-ideal 
devices: only a finite set of possible values is allowed for the signal amplitudes x[n] (see section 7.3). 

It should be clear from the above presentation that, for an arbitrary continuous-time signal x a (t), information 
will be lost through the uniform sampling process x a (t ) —> x\n] = x a (nT s ). 

This is illustrated by the following example. 
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*«(0 


► 


A/DT 


T 


x[n\ 

—► 


Fig. 7.3 Block diagram representation of ideal A/DT conversion. 


Example 7.1: Information Loss in Sampling 

► Consider the analog signals 

x a \ (t) = cos(2jrFif), F\ = 100 Hz 

x a i{t) = cos(27t F 2 t), F 2 = 300 Hz 

Uniform sampling at the rate F s = 200Hz yields 

x\ [n] = x c \ ( nT s ) = cos{^2%F\n / F s ) = cos(jtn) 

xi[n\ =x C 2 (nT s ) = cos(2nF 2 n/F s ) = cos(3roz) 

Clearly, 

x\ [n] =x 2 [n] 

This shows that uniform sampling is a non-invertible, many-to-one mapping. ◄ 

This example shows that it is not possible in general to recover the original analog signal x a (t) for all t G M, 
from the set of samples x[n\, n G Z. 

However, if we further assume that x a (t) is band-limited and select F s to be sufficiently large, it is possible 
to recover x a (t) from its samples x[n\. This is the essence of the sampling theorem. 


7.1.1 Frequency domain representation of uniform sampling 

In this section, we first investigate the relationship between the Fourier transform (FT) of the analog signal 
x a (t) and the DTFT of the discrete-time signal x\n] = x a (nT s ). The sampling theorem will follow naturally 
from this relationship. 


FT of analog signals: For an analog signal x a {t), the Fourier transform, denoted here as X a (Q.), £2 e M, is 
defined by 

/ oo 

x a (t)e- jn, dt (7.4) 

-oo 

The corresponding inverse Fourier transform relationship is given by: 

1 r°° 

xai!) = x- / X a (D.)e jQt dD. (7.5) 

2 K J- oo 

We assume that the student is familiar with the FT and its properties (if not, Signals and Systems should be 
reviewed). 
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*«(0 


x s (t) 

Impulse to 

x[«] 

* 

Sequence 



s(t) 


Fig. 7.4 Uniform sampling model. 


A sampling model: It is convenient to represent uniform sampling as shown in Figure 7.4. 
In this model: 


• s(t) is an impulse train, i.e.: 

oo 

s{t)= 52 &a(t-nT s ) (7.6) 

fl ——OO 

• x s (t) is a pulse-amplitude modulated (PAM) waveform: 


X 

V> 

II 

(7.7) 

= 52 x a (nT s )8 a (t — nT s ) 

77——oo 

(7.8) 

oo 

= 52 x[n]8 a (t - nT s ) 

(7.9) 


n——oo 


• The sample value x[n] is encoded in the amplitude of the pulse 8 a (t — nT s ). The information content 
of x s (t) and x[n\ are identical, i.e. one can be reconstructed from the other and vice versa. 


FT/DTFT Relationship: Let X(co) denote the DTFT of the samples x[n] = x a (nT s ), that is, X(co) = 
Zn^ x \n\e- jmn . The following equalities hold: 


X(ra)U=or,=X,(fl) 



X a (Q. — kGl s ) 


(7.10) 


Proof: The first equality is proved as follows: 


X S {Q) 




oo 

{ 52 x[n]b a (t-nT s )}e~ jSh dt 

n——oo 


52 x[n] / 8 a (t-nT s )e jat dt 

Yl =—oo J 00 


L x[n]e 

77——co 


j0.nT s 


X(to) at to = Q.T S 


(7.11) 
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Now consider the second equality: From ECSE 304, we recall that 

5(H) = FT{s(t)} 

oo 

= FT{ £ 5 a (t-nT s )} 

n——oo 

271 °° 

= — £ 8 a (Q-kQ s ) (7.12) 

s k=—oo 

Finally, since x s (t) = x a (t)s(t), it follows from the multiplicative property of the FT that 

x s (n) = ^-x a (a)*s(Q) 

271 

1 r°° 2 k 00 

= — £ 5.(11'-^)^ 

Z7l J —OO 1 c J 

= ^ £ X a (£L-k£t s ) □ (7.13) 

U k =-°o 

Remarks: The LF1S of (7.10) is the DTFT of x[n] evaluated at 

Q. F 

(0 = Q.T s = — =2n— (7.14) 

F F s 

We shall refer to y as the normalized frequency. The middle term of (7.10) is the FT of the amplitude 
modulated pulse train x s (t) = — nT s ). The RHS represents an infinite sum of scaled, shifted 

copies of X a (Q.) by integer multiples of <T S . 


Interpretation: Suppose x a (t) is a band-limited (BL) signal, i.e. X a (Q) = 0 for |H| > Q,y, where fl.v 
represents a maximum frequency. 


• Case 1: suppose £2 S > 2fly (see Figure 7.5). Note that for different values of k, the spectral images 
X a (Q. — kQ. s ) do not overlap. Consequently, X a (Q.) can be recovered from W(oo): 


X a (Q) 


T S X(£IT S ) |Q| < Q. s /2 

0 otherwise 


(7.15) 


• Case 2: suppose D. s < 2Qy (see Figure 7.6). In this case, the different images X a (Q — kQ s ) do overlap. 
As a result, it is not possible to recover X a (Q) from X (to). 

The critical frequency 2Fly (i.e. twice the maximum frequency) is called the Nyquist rate for uniform 
sampling. In case 2, i.e. Q. s < 2Fly, the distortion resulting from spectral overlap is called aliasing. In 
case 1, i.e. £l s > 2 fly. the fact that X a {Q.) can be recovered from W(co) actually implies that x a (t) can be 
recovered fromx[n]: this will be stated in the sampling theorem. 
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Fig. 7.5 Illustration of sampling in the frequency domain, when Q s > 2fly 


7.1.2 The sampling theorem 

The following theorem is one of the cornerstones of discrete-time signal processing. 


Sampling Theorem: 


Suppose that x a (t) is Band-Limited to |H| < Fly. Provided the sampling frequency 

LL = ^ > 2<T, V (7.16) 

then x a {t) can be recovered from its samples x[n] = x a [nT s ) via 


oo 

x a (t)= x[n]h r (t — nT s ), any t £ M 

n——oo 


(7.17) 


where 


A sin(7tr/r,) 
nt/T s 


sin c(t/T s ) 


(7.18) 


Proof: From our interpretation of the FT/DTFT relationship (7.10), we have seen under Case 1 that when 
a s > 2f2,y, it is possible to recover X a (Q.) from the DTFT X (to), or equivalently X s (£2), by means of (7.15). 
This operation simply amounts to keep the low-pass portion of X (£2). 
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T =0.026429, Cl =237.7421 




Fig. 7.6 Illustration of sampling in the frequency domain, when Cl s < 2fly 


This can be done with an ideal continuous-time low-pass filter with gain T s , defined as (see dashed line in 
Figure 7.5): 


H r (£l) 


T s |Q| < Q s /2 

0 |Q|>£V 2 


The corresponding impulse response is found by taking the inverse continuous-time Fourier Transform of 
H r (Q)\ 


1 r°° 1 /■ 

h r (t) = — / H r (n)e jat dD. = — / T s e jill d£l 
J —oo 271 J — 

T s e j& s t/2 _ e -jCl s t/2 

2n jt 

sin (nt/T s ) 
nt/T s 


Passing x s (t) through this impulse response h r (t) yields the desired signal x a (t), as the continuous-time 
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convolution: 


X a (t ) 


X s (t)*h r (t) 


/ Y § a (^ ~nTs)x a {nT s ) 

^ — 00 n——oo 


sin (7t (t-x)/T s ) 
n (t-i)/T s 


Y x a (nT s ) / 8 a (z-nTs) 


sin (7t(t — t.)/T s ) 
71 (t-T)/T s 


Y x a{ nT s ) 

n——oo 


sin (71 (t-nT s )/T s ) 
n(t — nT s )/T s 


the last equality coming from the fact that f^5a(t — to)f(t)dt = /(to)- □ 


Interpretation: 

The function h r (t) in (7.18) is called reconstruction filter. The general shape of h r (t) is shown in Figure 7.7. 
In particular, note that 

h r (mT s ) = 8 [m] = j * 

Thus, in the special case t = mT s , (7.17) yields 


m = 0 
otherwise 


(7.19) 


oo 

x a {mT s )= Y x[n]h r ((m-n)T s ) =x[m] 

ft——oo 


(7.20) 


which is consistent with the definition of the samples x[m]. In the case t / mT s , (7.17) provides an interpo¬ 
lation formula: 

x a (t) = Y scaled and shifted copies of sinc(t/7() (7.21) 

Equation (7.17) is often called the ideal band-limited signal reconstruction formula. Equivalently, it may be 
viewed as an ideal form of discrete-to-continuous (D/C) conversion. The reconstruction process is illustrated 
on the right of Figure 7.7. 


7.2 Discrete-time processing of continuous-time signals 

Now that we have seen under which conditions an analog signal can be converted to discrete-time without 
loss of information, we will develop in this section the conditions for the discrete-time implementation of a 
system to be equivalent to its continuous-time implementation. 

Figure 7.8 shows the structure that we will consider for discrete-time processing of analog signals x s (t). 
This is an ideal form of processing in which the amplitudes of DT signals x[n] and y[n] are not constrained. 
In a practical digital signal processing system, these amplitudes would be quantized (see section 7.3). 

The purpose of this section will be to answer the following question: Under what conditions is the DT 
structure of Figure 7.8 equivalent to an analog FTI system? 
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Fig. 7.7 Band-limited interpolation: Left, the general shape of the reconstruction filter h r (t) 
in time (T s = 1 unit); Right: the actual interpolation by superposition of scaled and shifted 
versions of h r (t). 


DT system 



Fig. 7.8 Setting for discrete-time processing of continuous-time signals. 


7.2.1 Study of input-output relationship 

Mathematical characterization: The three elements of Figure 7.8 can be characterized by the following 
relationships: 

• Ideal C/D converter: 

^(co)|«,=ar s = ^ £ XaiSl-kSls) (7.22) 

s k——oo 

where £l s = 2n/T s . 

• Discrete-time LTI filter: 

F(co) =//(co)X(co) (7.23) 

where H( to) is an arbitrary DT system function 

• Ideal D/C converter (see proof of sampling theorem): 

Y a (Q)=H r (Si)Y(QT s ) (7.24) 
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where H,fQ.) is the frequency response of an ideal continuous-time low-pass filter, with cut-off fre¬ 
quency £l s /2, i.e. 


H r (ii) 


T s |Q| < a s /2 
0 otherwise 


(7.25) 


Overall Input-output relationship: Combining (7.22)—(7.24) into a single equation, we obtain 

Y a (Q)=H r (Q)H(Qr s )^ £ X a (Q-kQ s ). (7.26) 

S k=~ oo 

Recall that for an analog LTI system, the Fourier Transforms of the input x a (t) and cones ponding output 
y a (t) are related through 

Y a {a) = G(Q)X a (a). (7.27) 

In general, (7.26) does not reduce to that form because of the spectral images X a (Q. — k£l s ) for k f 0. 
However, if we further assume that„r a (f) is band-limited to |0| < fly and that £2 S > 2 fly, then the products 

H r (D.)X a (D. - m s ) = 0, for all k^O. (7.28) 


In this case, (7.26) reduces to 


Yjn) = H(nT s )Xa(Q) 


which coiTesponds to an LTI analog system. 


(7.29) 


Theorem 1 (DT processing of analog signals) Provided that analog signal x a (t) is band-limited to F> < 
fi.v and that Q. s > 2fly, discrete-time processing of x\n] = x a (nT s ) with //(oo) is equivalent to analog pro¬ 
cessing of x a {t) with 


H a {n) 


h{q,t s ) if\a\<a s /2 

0 otherwise 


(7.30) 


Remarks: Note that H a (Q.) is set to zero for |H| > fl v /2 since the FT of the input is zero over that range, 
i.e. X a (Q) = 0 for |Q| > Q s /2 . The Theorem states that the two systems shown in Figure 7.9 are equivalent 
under the given conditions. 

In practice, many factors limit the applicability of this result: 

- input signal x a (t ) not perfectly band-limited 

- non-ideal C/D conversion 

- non-ideal D/C conversion 

These issues are given further considerations in the following sections. 


Example 7.2: 


► 


Consider a continuous-time speech signal limited to 8000 He. If it is desired to send this signal over a 
telephone line, it has to be band-limited between 300 He and 3400 He. The band-pass filter to be used is 
then specified as: 


Haim 


1 60071 < |fl| < 68 007T 

0 otherwise 
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DT LTI system 



*„(0 


analog LTI system 


> 


H a (0) 


> y,M) 


Fig. 7.9 Equivalence between Analog and Discrete-Time processing. 


If this filter is to be implemented in discrete-time without loss of information, sampling must take place 
above the Nyquist rate, i.e. with F s > 16 kHz, or Cl s > 3200071. If £2 S = 3200071 is chosen, the discrete¬ 
time filter which will implement the same band-pass filtering as H a (Cl) is given by: 

. _ f 1 600Tt/16000 < co < 6800Tt/16000 

( 0 otherwise in [—7t,7t], 

the specifications outside [—7t,7t] follow by periodicity of //(co). ◄ 


7.2.2 Anti-aliasing filter (AAF) 

Principle of AAF: 

The AAF is an analog filter, used prior to C/D conversion, to remove high-frequency content of x a (t) when 
the latter is not BL to ±D v /2. Its specifications are given by: 

Haam = { o 


| d | < a s /2 

otherwise 


(7.31) 


and are illustrated in Figure 7.10. 

The AAF avoids spectral aliasing during the sampling process, but is also useful in general even if the signal 
to be sampled is already band-limited: it removes high-frequency noise which would be aliased in the band 
of interest by the sampling process. 

The AAF is necessary, but has some drawbacks: if the signal extends over the Q v /2 limit, useful signal 
information may be lost; moreover, the direct analog implementation of H aa {Q.) with sharp cut-offs is costly, 
and such filters usually introduce phase-distortion near the cut-off frequency. 
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Fig. 7.10 Anti-aliasing filter template. 


Implementation issues: 

AAF are always present in DSP systems that interface with the analog world. Several approaches can be used 
to circumvent difficulties associated to the shatp cut-off requirements of the AAF filter H aa (D.) in (7.31). 
Two such approaches are described below and illustrated in Figure (7.11): 

• Approach #1: 

- Fix = maximum frequency of interest (e.g. Dy = 20kHz) 

- Set Q s = (1 + r)2Dy where 0 < r < 1 (e.g. r = 0.1 => Q s = 44kHz) 

- Use of non-ideal H aa (Q.) with transition band between F>,y and (1 +r)F>,y. In this case, a smooth 
transition band of width rDy is available. 

• Approach #2: M-times Oversampling: 

- Use cheap analog H aa (D.) with very large transition band 

- Significantly oversample x a (t) (e.g. Q s = 8 x 2Uy) 

- Complete the anti-aliasing filtering in the DT domain by a sharp low-pass DT filter H aa ( to), and 
then reduce the sampling rate. 

The second approach is superior in terms of flexibility and cost. It is generally used in commercial audio 
CD players. We will investigate it in further detail when we study multi-rate systems in chapter ??. 


7.3 A/D conversion 
Introduction 

Up to now, we have considered an idealized form of uniform sampling, also called ideal Analog-to-Discrete 
Time (AJDT) conversion (see Figure 7.12), in which the sampling takes place instantaneously and the signal 
amplitude x[n\ can take any arbitrary real value ,i.e. x[n] e M, so that x[n] is a discrete-time signal. 

A/DT conversion cannot be implemented exactly due to hardware limitations; it is used mainly as a mathe¬ 
matical model in the study of DSP. Practical DSP systems use non-ideal Analog-to-Digital (A/D) converters 
instead. These are characterized by: 

- Finite response time (the sampling is not quite instantaneous), 
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Fig. 7.11 Illustration of practical AAF implementations, (a) Excess sampling by factor 1 + r 
(b) Oversampling by factor M. 


analog signal 

A/DT 

Discrete-time signal 
x[n] = x a (nT s ) 




i 

k 

T. 



Fig. 7.12 Ideal A/DT Conversion. 

- Finite number of permissible output levels for x[n\ 

(i.e. x[n] e finite set =>• digital signal), 

- Other types of imperfections like timing jitter, A/D nonlinearities, ... 


7.3.1 Basic steps in A/D conversion 

Analog-to-Digital Conversion (A/D) may be viewed as a two-step process, as illustrated in Figure 7.13: 

• The Sample-and-Hold device (S&H) performs uniform sampling of its input at times nT s and main¬ 
tains a constant output voltage of x[n] until the next sampling instant. 

• During that time, the quantizer approximates x[n] with a value x[n] selected from a finite set of possible 
representation levels. 

The operation of each process is further developed below. 


Sample-and-hold (S&H): The S&H performs uniform sampling of its input at times nT s \ its output volt¬ 
age, denoted x{t), remains constant until the next sampling instant. Figure 7.14 illustrates the input and 
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A/D Converter 


x a (?) 


Sample 

& 

Hold 


x[»] 


-> Quantizer 


x[ri\ 

—► 


Fig. 7.13 Practical A/D Conversion 

output signals of the sample-and-hold device. 

Output 



Fig. 7.14 Illustration of a Sample-and-Hold operation on a continuous-time signal. 
Thus, the S&H output x(t) is a PAM waveform ideally given by 


oo 


x(t) 

= y, x \ n \ - nT s) 

n— —oo 

(7.32) 

x[n] 

E*? 

<3 

: X 

ii 

(7.33) 

P(t) 

f l ifo</<r, 

[ 0 otherwise 

(7.34) 


Practical S&H suffer from several imperfections: 

- The sampling is not instantaneous: x[n] ~ x a (nT s ); 

- The output voltage does not remain constant but slowly decays within a sampling period of duration 
/;• 
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Quantizer: The quantizer is a memoryless device, represented mathematically by a function Q(-), also 
called quantizer characteristic: 

x(n) = Q(x[n\). (7.35) 

Assuming a constant input voltage x[n], the function Q selects, among a finite set of K permissible levels, 
the one that best represents x[n\. 

x[n] £ M x[n\ £ FL = {xk}f=\ (7.36) 

where x k (k = 1 ,K) represent the permissible levels and M denotes the set of all such levels. In practice, 
B +1 bits are used for the representation of signal levels (B bits for magnitude + 1 bit for sign), so that the the 
total number of levels is K = 2 ri 1 . Since the output levels of the quantizer arc usually represented in binary 
form, the concept of binary coding is implicit within the function < 2(0 (more on this in a later chapter). 

For an input signal x[n\ to the quantizer, the quantization error is defined as 

e[n] =x[n] —x[n] (7.37) 

Uniform Quantizer: The most commonly used form of quantizer is the uniform quantizer, in which the 

representation levels are uniformly spaced within an interval [—Xf s ,Xf s ], where Xf s (often specified as a 
positive voltage) is known as the full-scale level of the quantizer. Specifically, the representation levels arc 
constrained to be 

x k = -X fs + (k~l)A, k=l,...,K. (7.38) 

• The minimum value representable by the quantizer is x mm = x\ = — Xf s 

• A, called the quantization step size, is the constant distance between representation levels. In practice, 
we have 

A = ^ = 2- B X k (7.39) 

K 

• The maximum value representable by the quantizer is x max = xk = Xf s — A 

• The dynamic range of the uniform quantizer is defined as the interval 

DR = [x l -^,x K + ^\ (7.40) 

While several choices are possible for the non-linear quantizer characteristic < 2(0 i n (7-35), a common 
approach is to round the input to the closest level: 



Xl 

if x[n 

<w-f 


Q(x[n ]) = < 

x k 

if x k - 

| < x[n] < x k + | for k = 1,.... K 

(7.41) 


©K 

if x[n 

>x K + f • 



Figure 7.15 illustrates the input-output relationship of a 3-bit uniform quantizer. In this case, K = 8 different 
levels exist. 

For the uniform quantizer, the quantization error e[n\ satisfies: 

x[n\ £ DR \e[n\\< A/2 (7.42) 

i[n] ^ DR =>• \e[n] \ > A/2 (clipping) 
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Fig. 7.15 3-bit Uniform Quantizer Input-output characteristic. The reproduction levels are 
labelled with their two’s complement binary representation. 


7.3.2 Statistical model of quantization errors 

In the study of DSP systems, it is important to know how quantization errors will affect the system output. 
Since it is difficult to handle such errors deterministically, a statistical model is usually assumed. A basic 
statistical model is derived below for the uniform quantizer. 


Model structure: From the definition of quantization error: 

e[n] = x[n\ — x[n\ =>x[n] = x[n] +e[n], (7.43) 

one can derive an equivalent additive model for the quantization noise, as illustrated in Figure 7.16. The 

e[n] 


x\n\ 


GO 


,r[/7] 

—► 


x\ri\ 


* 0 - 


x\n\ 

-> 


Fig. 7.16 Additive noise model of a quantizer 

non-linear device Q is replaced by a linear operation, and the error e[n] is modelled as a white noise sequence 
(see below). 


White noise assumption: The following assumptions are made about the input sequence x[n] and the 
corresponding quantization error sequence e[n ]: 
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- e[n\ and x[n] are uncorrelated, i.e. 

E{x[n]e[m]} = 0 for any n,m E Z 


where E{ } denotes expectation. 

- e[n\ is a white noise sequence, i.e.: 

E\e[n]e[m]} = 0 if m ^ n 

- e[n\ is uniformly distributed within ±A/2, which implies 


mean 

variance 


E{ e [n]} = 0 



These assumptions are valid provided the signal is sufficiently “busy”, i.e.: 

- a x = RMS value of x[n] S> A; 

- x[n\ changes rapidly. 


(7.44) 


(7.45) 


(7.46) 

(7.47) 


SQNR computation: the signal to quantization noise ratio (SQNR) is a good characterization of the effect 
of the quantizer on the input signal. It is defined as the ratio between the signal power and the quantization 
noise power at the output of the quantizer, in dB: 

G 2 

SQNR = 101og 10 -|, 

where a x and a e represent the RMS value of the signal and noise respectively. With the above assumption 
on the noise, we have that 

2 = A ^ = 2- 2 *4 
e 12 12 ’ 

where Xf s is the full scale level of the quantizer, as illustrated in Figure 7.15, so that 

SQNR = 201og 10 ^ + 101og 10 12 + 20£log 10 2 

^fs 


or finally 


SQNR = 6.02 B + 10.8 - 201og 10 — 

Ox 


(dB) 


(7.48) 


The above SQNR expression states that for each added bit, the SQNR increases by 6 dB. Moreover, it 
includes a penalty term for the cases where the quantizer range is not matched to the input dynamic range. 
As an example, if the dynamic range is chosen too big (Xf s 3> o x ), then this term will add a big negative 
contribution to the SQNR, because the usage of the available quantizer reproduction levels will be very 
bad (only a few levels around 0 will be used). This formula does not take into account the complementary 
problem (quantizer range too small, resulting in clipping of useful data), because he underlying noise model 
is based on a no-clipping assumption. 
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7.4 D/A conversion 

Figure 7.17 shows the ideal discrete time-to-analog conversion (DT/C) as considered up to now. It uses an 
ideal low-pass reconstruction filter h r (t), as defined in equation (7.18). The corresponding frequency and 

DT signal Analog signal 


vM 

DT/A 

T,(0 




i 

i, 

T s 



Fig. 7.17 Ideal Discrete Time-to-Analog Conversion 


time input/output expressions are: 

and 

where 


oo 

y r (t)= Yj y[n\h r (t-nT s ) 

n=—oo 


Y r (Q) = Y(£2T s )H r (Q) 


H r (Q) 


T s |n|<Q,/2 

0 otherwise 


(7.49) 

(7.50) 

(7.51) 


This is an idealized form of reconstruction, since all the signal samples y[n] are needed to reconstruct y r {t). 
The interpolating function h r (t) corresponds to an ideal low-pass filtering operation (i.e. non-causal and 
unstable). 


Ideal DT/A conversion cannot be implemented exactly due to physical limitations; similarly to A/DT, it is 
used as a mathematical model in DSP. Practical DSP systems use non-ideal D/A converters instead charac¬ 
terized by: 

- a realizable, low-cost approximation to h r (t). 

- the output signal y r (t) is not perfectly band-limited. 


7.4.1 Basic steps in D/A conversion: 

D/A may be viewed as a two-step process, illustrated in Figure 7.18: 

• The hold circuitry transforms its digital input into a continuous-time PAM-like signal y(t); 

• The post-filter smoothes out the PAM signal (spectral reshaping). 


Hold circuit: A Time-domain description of its operation is given by: 

oo 

y(t) = Y y[n\h 0 {t - nT s ) (7.52) 

n——oo 
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D/A converter 



Fig. 7.18 Two-step D/A Conversion. 


where / 20 (f) is the basic pulse shape of the circuit. In the frequency domain: 


Yr(&) 


/ oo 

y(t)e~i at dt 

-00 

OO r, OQ 

£ y[?t] / ho(t— nTs)e~^ t dt 

n——oo J ~ 00 

00 />oo 

£ / h 0 {t)e~ jQi dt 

n =-00 */— 00 

y(nr,)tf 0 (o) 


(7.53) 


In general, Hq(£1) / H r (£l), so that the output y(f) of the hold circuit is different from that of the ideal DT/A 
conversion. The most common implementation is the zero-order hold, which uses a rectangular pulse shape: 



if 0 < t < T s 
otherwise 


(7.54) 


The corresponding frequency response is 

Hq(£ 1) = T s sinc^/n,) g-Z^/O. (7.55) 


Post-filter: The purpose of the post-filter H p f(D.) is to smooth out the output of the hold circuitry so that 
it resembles more that of an ideal DT/A function. 

From the previous discussions, it is clear that the desired output of the D/A converter is 

Y r (Sl)=Y(QT s )H r (Sl), 


whereas the output of hold circuitry is: 

y(£2) = y(nr,)tf 0 (o)- 


Clearly, the output of the post filter will be identical to that of the ideal DT/A converter if 


H pf (n) = H r {ei)/Ho(n) 


T S /H 0 (D) if \Q.\ < Q. s /2 
0 otherwise 


In practice,the post filter H p f(Q.) can only be approximated but the results are still satisfactory. 


(7.56) 
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Example 7.3: Zero-order hold circuit 

► The zero-order hold frequency response is shown on the top of Figure 7.19, as compared to the ideal 
brick-wall low-pass reconstruction filter H r (D.). The compensating filter in this case is shown in the 
bottom part of Figure 7.19. ◄ 




Fig. 7.19 Illustration of the zero-order hold frequency response (Top, compared with ideal 
low-pass interpolation filter), and the associated compensation post-filter (bottom ).(T denotes 
the sampling period) 
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Chapter 8 

Structures for the realization of DT systems 

Introduction: 

Consider a causal LTI system Oi with rational system function: 

H(z) = Z{h[n}} = ^. (8.1) 

In theory, H(z) provides a complete mathematical characterization of the system ‘M. However, even if we 
specify H(z), the input-output transformation x[n] —>y[n] = ‘.H{x{n\ }, where x\n\ denotes and arbitrary input 
and y[n] the corresponding output, can be computed in a multitude of different ways. 

Each of the different ways of performing the computation y[n] = 9{{x[n]} is called a realization of the 
system. Specifically: 

• A realization of a system is a specific (and complete) description of its internal computational struc¬ 
ture. 

• This description may involve difference equations, block diagrams, pseudo-code (Matlab like), etc. 

For a given system tH, there is an infinity of such realizations. The choice of a particular realization is 
guided by practical considerations, such as: 

• computational requirements; 

• internal memory requirements; 

• effects of finite precision representation; 

• chip area and power consumption in VLSI implementation; 

• etc. 

In this chapter, we discuss some of the most well-known structures for the realization of FIR and HR discrete¬ 
time systems. We will use Signal Flow-Graph representation to illustrate the different compuational struc¬ 
tures. These are reviewed in the next section. 
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8.1 Signal flow-graph representation 

A signal flow-graph (SFG) is a network of directed branches connecting at nodes. It is mainly used to 
graphically represent the relationship between signals. It in fact gives detailed information on the actual 
algorithm that is used to compute the samples of one of several signals based on the samples of one or 
several other signals. 

Basic terminology and operation principles of SFG are detailed below: 


Node: A node is represented by a small circle, as shown in Figure 8.1(a). To each node is associated a 
node value, say w[n], that may or not appear next to the node. The node value represents either an external 
input, the result of an internal computation, or an output value (see below). 


Branch: A branch is an oriented line segment. The signal flow along the branch is indicated by an arrow. 
The relationship between the branch input and output is provided by the branch transmittance, as shown in 
Figure 8.1(b). 


vi [n] 
o 


(a) 


input output 

u[n ] v[n\ 



H{z) 

► 


(b) 


v[n] = u[n] 
v[n] = au[n] 
v[n ] = u[n-l] 
V(z) = H(z)U(z) 


Fig. 8.1 Basic constituents of a signal flow-graph: (a) a single isolated node; (b) different 
branches along with their input-output characteristic. 


Internal node: An internal node is a node with one or more input branches (i.e. entering the node) and 
one or more output branches (i.e. leaving the node), as shown in Figure 8.2(a). 

The node value w[n] (not shown in the figure) is the sum of the outputs of all the branches entering the node, 
i.e. 

K 

w[n\=Y^Uk[n\. ( 8 . 2 ) 

k= l 

The inputs to the branches leaving the node are all identical and equal to the node value: 

vi [n] = ■■■ = v L [n\ = w[n\. (8.3) 
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Fig. 8.2 Different types of nodes: (a) internal node; (b) source node; (c) sink node. 


Source node: A source node is a node with no input branch, as shown in Figure 8.2(b). The node value 
x[n\ is provided by an external input to the network. 


Sink node: A sink node is a node with no output branch, as shown in Figure 8.2(c). The node value y[n], 
computed in the same way as an internal node, may be used as a network output. 


Remarks: SFGs provides a convenient tool for describing various system realizations. In many cases, 
non-trivial realizations of a system can be obtained via simple modifications to its SFG. 

In the examples below, we illustrate the use of SFG for the representation of rational LTI systems. In 
particular, we illustrate how a SFG can be derived from an LCCDE description of the system and vice versa. 


Example 8.1: 

► Consider the SFG in figure 8.3, where a and b are arbitrary constants: 

M 



Fig. 8.3 SFG for example 8.1. 

The node value can be expressed in terms of input signals x\ [n] and r 2 [n] as 

w[n] =x\[n— 1] +ax 2 [n] 
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From there, an expression for the output signal y[n] in terms of x\ [n] and X 2 [«] is obtained as 

y[n] = bw[n] 

= bxi\n—\\+ abx 2 [n] 

◄ 


Example 8.2: 


► Consider the system function 


H(Z) = 


bo + biz 1 


1 — az 1 

To obtain a SFG representation of H(z), we first obtain the corresponding LCCDE, namely 

y\n] = ay[n — 1] + box[n] + b\x[n — 1] 


The derivation of the SFG can be made easier by expressing the LCCDE as a set of two equations as 
follows: 


w[n] = b(,x[n]+b\x[n—\] 

y\n] = ay[n—\\+w[n] 

From there, one immediately obtains the SFG in figure 8.4. 


x[n] h w[n] 

w_hi!_ 


-►-< 

r--S 

r-- c 

r--9 

-1 




Z 1 

' 

L i 

L ” 


h 


a 


)--c 

5 o- * -0 


ay[n -1] 



Fig. 8.4 SFG for example 8.2 

Note the presence of the feedback loop in this diagram which is typical of recursive LCCDE ◄ 


Example 8.3: 

► Let us find the system function H(z.) corresponding to the SFG in figure 8.5 

In this type of problem, one has to be very systematic to avoid possible mistakes. We proceed in 4 steps 
as follows: 

(1) Identify and label non-trivial internal nodes. Here, we identify two non-trivial nodes, labelled 
w i [n] and vv ’2 [n] in Figure 8.5. 

(2) Find the “-domain input/output (I/O) relation for each of the non-trivial nodes and the output 
node. Here, there are 2 non-trivial nodes and 1 output node, so that we need a total of 3 linearly 


independent relations, namely: 



Y(z) 

= boW\(z) +b\W 2 (z) 

(8.4) 

W 2 (z) 

= z~ l Wi(z) 

(8.5) 

Wi(z) 

= X(z)+aW 2 {z) 

(8.6) 
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n] w x [n] b 0 

? r A 


-1 



i 

i z ' 

’ i 

L 

c 

a 

!>--c 

V-/ 

_A 

5 


w 2 [n] 


Fig. 8.5 SFG for example 8.3. 


(3) By working out the algebra, solve the I/O relations for Y (z) in terms of X (z). Here: 

(8.5)+ (8.6) =+ 

=+ Wi(z) = 


W\ (z) = X (z) + az 'W'i(z) 

X(z) 


1 — az 


(8.7) 


(8.4)+ (8.5)+ (8.7) =+ Y(z) = {bp+b\z~ l )Wi(z) 


( 8 . 8 ) 


(4) Finally, the system function is obtained as 


H(z) = 


y(z) _ bp + biz 1 

X (z) 1 - az ~ 1 


As a final remark, we note that the above system function is identical to that considered in Example 8.2, 
even tough the SFG in considered here is different from that in Example 8.3. ◄ 


8.2 Realizations of HR systems 


We consider HR systems with rational system functions: 

= Ztohz- k ± B(z) 


(8.9) 


The minus signs in the denominator are introduced to simplify the Signal Flow-Graphs (be careful, they are 
often a source of confusion). 

For this general HR system function, we shall derive several SFGs that correspond to different system 
realizations. 


8.2.1 Direct form I 

The LCCDE corresponding to system function H(z) in (8.9) is given by: 

N M 

y[n] = £ a k y[n - k\ + £ h*[ n ~ k] 
k= 1 k=0 


( 8 . 10 ) 
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x\n\ 
o-► 


z 't 


■O' 

n 




y[n] 


* -o 


b M- 1 


a N- 1 




" i 

1 i 

l 

b M 


a N 

0- ► - c 


5 - *l - 6 


B(z ) 1 /A(z) 

Fig. 8.6 Direct Form I implementation of an HR system. 


Proceeding as in Example 8.2, this leads directly to the SFG shown in Figure 8.6, known as direct form I 
(DFI). 

In a practical implementation of this SFG, each unit delay (i.e. z ') requires one storage location (past 
values need to be stored). 

Finally, note that in Figure 8.6, the section of the SFG labeled B(z) corresponds to the zeros of the system, 
while the section labelled 1 / A(z) corresponds to the poles. 


8.2.2 Direct form II 

Since FTI systems do commute, the two sections identified as B(z) and 1/A(z) in Figure 8.6 may be inter¬ 
changed. This leads to the intermediate SFG structure shown in Figure 8.7. 

Observe that the two vertical delay lines in the middle of the structure compute the same quantities, namely 
w[n — l],w[« — 2],..., so that only one delay line is actually needed. Eliminating one of these delay lines 
and merging the two sections lead to the structure shown in Figure 8.8, known as the direct form II (DFII). 

Note that (V = M is assumed for convenience, but if it is not the case, branches corresponding to a-, = 0 or 
bi = 0 will simply disappear. 

The main advantage of DFII over DFI is its reduced storage requirement. Indeed, while the DFI contains 
N+M unit delays, the DFII only contains max{fV,M} < N + M unit delays. 

Referring to Figure 8.8, one can easily see that the following difference equations describe the DFII realiza- 
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w[n\ b 0 
o-►-o- 



y[n\ 

-o 


a N-\ 


b M -\ 




n 1 

'z 1 z 

' A 

a N 



n 

O 

5 C 

)-►-6 


\IA(z) B{z ) 

Fig. 8.7 Structure obtained from a DFI by commuting the implementation of the poles with 
the implementation of the zeros. 


tion: 


w[n] = a\w[n— 1] H- ciNw[n — N] +x[n] (8.11) 

y[n] = bow[n] +b\w[n — 1] H- \-bMw[n — M] (8.12) 


8.2.3 Cascade form 

A cascade form is obtained by first factoring H{z) as a product: 

H{z) = YTi=H i (z)H 2 {z) .. ,H K (z ) (8.13) 

A(z) 

where 

Hk (z) = j^, k=l,2,~,K (8.14) 

represent HR sections of low-order (typically 1 or 2). The corresponding SFG is shown in Figure 8.9. Note 
that each box needs to be expanded with the proper sub-SFG (often in DFII or transposed DFII). 
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x[n\ n[n\ b 0 y[n] 


-►-c 

r -- < 

r -*- s 

r-*"- 

i 

L ' 

r z _1 i 

1 


a , 

b i 



)- M -( 

)-►-< 



a N- 1 

b N- 1 



L ' 

r z 1 i 

a N 

b N 


6 - * - 6 -►- 6 


Fig. 8.8 Direct Form II realization of an HR system. 



Fig. 8.9 Cascade Form for an HR system. Note that each box needs to be expanded with the 
proper sub-SFG (often in DFII or transposed DFII). 


For a typical real coefficient second-order section (SOS): 


v i ^ (1-Z*1Z )(1“Z*2Z ) 

M k{Z) = O* _ 1W1 _ n 

{1-piclZ ^(l ~PklZ *) 

(8.15) 

bkO + bk 1 1 +bk2Z~ 2 

(8.16) 

1 -a k iz~ l -a k2 z~ 2 


where p k 2 = p' k \ and Zki = z* kl so that tiki-bki £ M. It should be clear that such a cascade decomposition of 
an arbitrary H[z) is not unique: 

• There are many different ways of grouping the zeros and poles of H(z) into lower order sections; 

• The gains of the individual sections may be changed (provided their product remains constant). 


Example 8.4: 

► Consider the system function 

(1 - 1 ) (1 - V 1 ) (1 - e^/ A z~ 1 ) (1 - e^V 1 ) 

(1 - .9z _1 )(l + .9z _1 )(l — .9y'z _1 )(l + -9 jz~ l ) 
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There are several ways of pairing the poles and the zeros of H[z) to obtain 2nd order sections. Whenever 
possible, complex conjugate poles and zeros should be paired so that the resulting second order section 
has real coefficient. For example, one possible choice that fulfils this requirement is: 

( 1 - e^/V 1 ) ( 1 - e-;*/V 1 ) 1 - sjlz~ 1 + z~ 2 

(1 — .9z —1 ) (1 T .9z~ 1 ) “ l-.81z- 2 

, . _ (1 — e- , ’ 3,t / 4 z _1 )(l —e~- , ' 3,t / 4 z -1 ) ^ 1 + v/2z —1 + 

2 ~ (1 — .97'z -1 )(1 + -9jz~ l ) l + .81z -2 

The corresponding SFG is illustrated in figure 8.10 where the above 2nd order sections are realized in 
DFII. Note that we have arbitrarily incorporated the gain factor G in between the two sections II \ (z) and 


x[«] 

o-► 


0 -►-f 

r--? 

l ’ 

r Z _1 n 


-V2 

O £ 

r- * -y 

L 

’Z~' n 

.81 


6--c 

3-►-O 


G 


y[n] 


0 -►-c 

)-►-c 

r-- 

JL ' 

’ z~ l 

1 


V2 


0 <! 

P - - c 

p 

n 

r z _1 

L 

-.81 



6-x-c 

>-►-c 

3 


v 


"V 


H 2 {z) 


Fig. 8.10 Example of a cascade realization 

H 2 (z). In practice, the gain G could be distributed over the SFG so as to optimize the use of the available 
dynamic range of the processor (more on this later). ◄ 


8.2.4 Parallel form 


Based on the partial fraction expansion of H(z), one can easily express the latter as a sum: 

H(z)=C{z) + f j H K {z) 
k =1 

H k (z) : low-order HR sections 
C(z) ■ FIR section (if needed) 


(8.17) 

(8.18) 
(8.19) 


The corresponding SFG is shown in Figure 8.11, where each box needs to be expanded with a proper sub- 
SFG. Typically, second-order HR sections H k (z) would be obtained by combining terms in the PFE that 
correspond to complex conjugate poles: 


Hk{z) 


1 -PkZ 1 + 1 - p* k Z 1 
bko + b k iz^ 1 
1 -a k \z~ x -a k2 z~ 2 


( 8 . 20 ) 

( 8 . 21 ) 
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x\n\ 

o— 


o C(z) o 


o H x {z) o 


<>H 2 (z)i> 


° H k (z) (> 


4? 1 


Fig. 8.11 Parallel Realization of an HR system. 


where £ M. A DFII or transposed DFII structure would be used for the realization of (8.21). 


Example 8.5: 


► Consider the system function in Example 8.4 

(1 - e^ 4 z~ 1 ) (1 - 1 ) (1 - ^ 3 */ 4 z" 1 ) (1 - e~W 4 z~ 1 ) 

(1 — .9z _1 )(l + .9z _1 )(l — .97 'z - 1 )(1 + .9yz _1 ) 


It is not difficult to verify that it has the following PFE: 


B 


C 


H(z) — A + - -—- r + --—-r + 

1 — .9z~ x 1 + .9z~ l 


D 


1 - .9 jz 


D* 

1 + .9 jz~ l 


where A, B and C are appropriate real valued coefficients and D is complex valued. Again, there are 
several ways of pairing the poles of H(z) to obtain 2nd order sections. In practice, terms corresponding 
to complex conjugate poles should be combined so that the resulting second order section has real 
coefficients. Applying this approach, we obtain 


Hi(z) = 


B 

1-.9Z- 1 


C _ bio + bnz 1 
1 + .9z- 1 ~ l-.81z“ 2 


Hi(z) = 


D 


D* 


bzo + bziz 


1 — .9 jz~ l 1 + .9 jz- 1 l + .81z- 2 

The corresponding SFG is illustrated below with Hi (z) and Hz (z) realized in DFII: 


◄ 


8.2.5 Transposed direct form II 
Transposition theorem: 

Let C]\ denote a SFG with system function H\(z). Let Qi denote the SFG obtained from tjj by: 
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A 



Fig. 8.12 Example of a parallel realization 


(1) reversing the direction of all the branches, and 

(2) interchanging the source x[n\ and the sink y[n]. 

Then 

H 2 (z)=H ] (z) (8.22) 

Note that only the branch direction is changed, not the transmittance. Usually, we redraw (j 2 so that the 
source node x[n] is on the left. 


Example 8.6: 


► 


An example flowgraph is represented on the left of Figure 8.13. Its transfer function is easily found to 
be 

1 —abz 1 


The application of the transposition theorem yields the SFG on the right of Figure 8.13, which can easily 
be checked to have the same transfer function. ◄ 


Application to DFII: 

Consider the general rational transfer function 

H (z)= L h hZ k k - ( 8 - 23 ) 

1-L k=\ a kZ k 
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x[n\ h z' 1 y[«] 


) % 1 

'z 1 

c ' 





Fig. 8.13 Example of application of the transposition theorem: left, original SFG and right, 
transposed SFG. 


with its DFII realization, as shown in Figure 8.8. Applying the transposition theorem as in the previous 
example, we obtain the SFG shown in figure 8.14, known as the transposed direct form II. Note that N = M 
is assumed for convenience. 


x[n] b 0 y[n] 




Fig. 8.14 Transposed DFII realization of a rational system. 


The corresponding LCCDEs are : 


w N [n\ 

= b N x[n\+a N y[n\ 

w N -i[n] 

= b N -ix[n]+a N -iy[n]+w N [i 

w\[n\ 

= bix[n]+a\y[n} + W 2 [n — \\ 

y[n] 

= w\[n- 1 ] +b 0 x[n] 


(8.24) 
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8.3 Realizations of FIR systems 

8.3.1 Direct forms 

In the case of an FIR system, the denominator in the rational system function (8.9) reduces to A(z) = 1. That 
is 


M 

H(z)=B(z)='£b k z- k , 


k=0 


(8.25) 


corresponding to the following non-recursive LCCDE in the time-domain: 


M 

y[n\ = Yj b kx[n - k\ 


k =o 


(8.26) 


Accordingly, the DFI and DFII realizations become equivalent, as the structural component labelled 1 /A{z) 
in Figure 8.6 disappears. The resulting SFG is shown in Figure 8.15; it is also called a tapped delay line or 
a transversal filter structure. 


x[n\ 
o-► 


o-►-c 

>- ■ • • - c 

j> - ► - o 

bn 

b 


O0 1 

' 1 ' 

r U M -1 i 

V / 

it 

— o 

!>- - d 

V f 

U 

o — 


vM 

-o 


Fig. 8.15 DF for an FIR system (also called tapped delay line or transversal filter). 


Note that the coefficients bk in Figure 8.15 directly define the impulse response of the system. Applying a 
unit pulse as the input, i.e. x[n\ = S[n], produces an output y[n] = h[n], with 


h [n] 


b n if n E {0,1,... ,M} 
0 otherwise. 


(8.27) 


© B. Champagne & F. Labeau 


Compiled October 13, 2004 

















160 


Chapter 8. Structures for the realization of DT systems 


8.3.2 Cascade form 

In the case of an FIR system, a cascade form is also possible but the problem is in fact simpler than for HR 
since only the zeros need to be considered (because A(z) = 1). 

To decompose H(z) in (8.25) as a product of low-order FIR sections, as in 

H(z)=H l (z)H 2 {z)---H K (z) 

one needs to find the zeros of H(z), say zi (l = 1, • • • ,M), and form the desired low-order sections by properly 
grouping the corresponding factors 1 — ziz ~ 1 and multiplying by the appropriate gain term. For example, a 
typical 2nd order section would have the form 


H k (z) = G k (l -z kl z *)(1 -Zk 2 z *) = b k0 + biaz l +b k2 z 2 . 

where the coefficients b k i can be made real by proper choice of complex conjugate zeros (i.e. z kl = z k ). 

In practice, cascade realizations of FIR system use sections of order 1, 2 and/or 4. Sections of order 4 are 
useful in the efficient realization of linear-phase system... 


Example 8.7: 

► Consider the FIR system function 

H{z) = (1 - e^ 4 z ~ 1 ) (1 - 1 ) (1 - e^ 4 z ~ 1 ) (1 - e~^ 4 z ~ 1 ) 

A cascade realization of H(z) as a product of 2nd order sections can be obtained as follows: 

H\(z) = (1 — 1 — e~i 7l l 4 z~ 1 ) = 1 - V2\zT x + z~ 2 

H 2 (z) = ( 1 - e^ 4 z ~ 1 ) (1 - e~^ 4 z ~ 1 ) = 1 + sflz~ 1 +z~ 2 


8.3.3 Linear-phase FIR systems 

Definition: A LTI system is said to have generalized linear-phase (GLP) if its frequency response is ex¬ 
pressible as 

//(go) =A((o)e~ j ^ a(0 ^ ) (8.28) 

where the function A (go) and the coefficients a and [3 are real valued. 

Note that A(co) can be negative, so that in general, A (go) / |//(co)|. In terms of A(co), a and P, we have 


IH{ go) = 


-occo + p if A (go) > 0 

—aco + P + 7i ifA(co)<0 


(8.29) 


Despite the phase discontinuities, it is common practice to refer to //(go) as a linear phase system. 
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Theorem: Consider an LTI system di with FIR h\n], such that 

h\n] = 0 for n < 0 and for n> M 
h[ 0] / 0 and h[M\ ± 0 

System di is GLP if and only if h [n] is either 

(i) symmetric, i.e.: 

li [n] = li [M — n ], n € Z 

(ii) or anti-symmetric, i.e.: 

h[n) = — h[M — n], n € Z 

Note that the GLP conditions (8.31) and (8.32) can be combined into a single equation, namely 

h[n\ = eh[M — n] 

where e is either equal to +1 (symmetric) or —1 (anti-symmetric). 


Example 8.8: 


► Consider the LTI system with impulse response 

A[n] = {1,0,1,0,1} 

Note that h[n) satisfies the definition (8.31) of a symmetric impulse response with M = 4: 

h[n] = h[4 — n], all n £ Z 

Let us verify that the system is linear phase, or equivalently, that its frequency response verifies the 
condition (8.28). We have 


H( co) = £ h[n]e~ im 

n=—oo 

- l+e-W + e-J 4 ' 0 
= e - j2t0 (e j2 '°+\+e- j2ta 
= e ~( 2co (l +2cos(2co)) 

= A(co)e“ 2 ( cl03 +l 3 ) 

where we identify 

A(co) = 1 +2cos(2co), 

a = 2, (3 = 0 


(8.30) 

(8.31) 

(8.32) 

(8.33) 
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Modified direct form: The symmetry in h[n\ can be used advantageously to reduce the number of multi¬ 
plications in direct form realization: 

h[n] = eh[M — n\ => b\ = zbM-k (8.34) 

In the case M odd, we have 

M 

y[n\ = Y b k x[n - k] (8.35) 

k= 0 

(M~ l)/2 

= Y ( bkx[n ~k} + b M -kx[n - (M - £)]) 
k =o 

(M- 1)/2 

= Y bk(x[n — k\ + ex[n — (M — k)]) 
k= 0 

which only requires ( M + l)/2 multiplications instead of M + 1. 

Figure 8.16 illustrates a modified DFI structure which enforces linear phase. 


x[n] 



Fig. 8.16 A modified DF structure to enforce linear phase (M odd). 


Properties of the zeros of a GLP system: In the z-domain the GLP condition h\n] = eh[M — n] is equiv¬ 
alent to 

H{z) = ez- M H{ z - 1 ) (8.36) 

Thus, if zo is a zero of H(z), so is 1 /zo- If in addition h[n\ is real, then zo, Zq, 1 /zo and 1 /zq are all zeros of 
//(z). This is illustrated in Figure 8.17. 

As a result, sections of order 4 are often used in the cascade realization of real, linear phase FIR filters: 

H k (z) = G k (\-ZkZ- x ){\-zlz~ x ){ 1 - V‘)( 1 - ^z” 1 ) (8.37) 

Zk Zfc 


8.3.4 Lattice realization of FIR systems 

Lattice stage: The basic lattice stage is a 2-input 2-output DT processing device. Its signal flow graph is 
illustrated in Figure 8.18. 
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Fig. 8.17 Illustration of the zero locations of FIR real-coefficient GLP systems: a zero always 
appears with its inverse and complex conjugate. 


u m [n] 



Fig. 8.18 Lattice stage. 


© B. Champagne & F. Labeau 


Compiled October 13, 2004 


















164 


Chapter 8. Structures for the realization of DT systems 


The parameter K„, is called reflection coefficient, while m is an integer index whose meaning will be ex¬ 
plained shortly. 

Input-output characterization in the time domain: 


u m [n\ = u m -i [n] + K m v m _i [n — 1] 
v m [n\ = K* m u m -i [n] + v OT _i [n — 1] 


Equivalent z-domain matrix representation: 

Urn— l (^) 
E/n— 1 (v) 

where K m (z) is a 2 x 2 transfer matrix defined as 

K m {z)± \ \ -r 


U m (z) 

V m (v) 


= K m (z) 


(8.38) 

(8.39) 


(8.40) 


(8.41) 


Lattice filter: A lattice filter is made up of a cascade of M lattice stages, with index m running from 1 to 
M. This is illustrated in figure 8.19. 


«o [«1 “it'd m 2 [»] “mW y\n 1 



v 0 [«] v,[n] v 2 [h] v m [h] 


Fig. 8.19 A lattice filter as a series of lattice stages. 

CoiTesponding set of difference equations characterizing the above SFG in the time domain: 

uo[n] = v 0 [n] =x[n] 
for m = 1,2 ,... ,M 

u m [n\ = u m —\ [n] + [n - 1] 

v m [n) = K* m u m -i[n] +v m _i[n — 1] 

end 

y[n] = u M [n] 


Corresponding matrix representation in the z-domain: 


Uo(z) 
U M (z ) ' 
V M {v) _ 

r(z) 


Vb(z) = 

X(z) 


(8.42) 

K m (z) • 

■ ■ K 2 [z)K\ (z) 

' U Q (z) ' 
. V 0 (v) 

(8.43) 

U M (z) 



(8.44) 
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System function: 

• From (8.42)- (8.44), it follows that Y(z) 
by 

H(z) = [ 1 


= H(z)X(z) with the equivalent system function H(z) given 


0 ]K M (z)...K 2 (z)K x (z) 


(8.45) 


• Expanding the matrix product in (8.45), it is easy to verify that the system function of the lattice filter 
is expressible as 

H(z) = l+b 1 z~ 1 + ---+b M z- M (8.46) 

where each the coefficients b * in (8.46) can be expressed algebraically in terms of the reflection 
coefficients {k m }m=i 

• Conclusion: an M- stage lattice filter is indeed FIR filter of length M + 1. 


Minimum-phase property: The lattice realization is guaranteed to be minimum-phase if the reflection 
coefficients are all less than one in magnitude, that is: if |k,„| < 1 for all m = I ,_ M. 

Note: Thus, by constraining the reflection coefficients as above, we can be sure that a causal and stable 
inverse exists for the lattice system. The proof of this property is beyond the scope of this course. 
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Chapter 9 

Filter design 


9.1 Introduction 


9.1.1 Problem statement 


Consider the rational system function 


m 


m 

A(z) 


Lk=ohz- k 

1+Eti a kZ ~ k 


(9.1) 


for which several computational realizations have been developed in the preceding Chapter. 

In filter design, we seek to find the system coefficients, i.e. M,N,a \,... ,on, bo,... ,Bm, in (9.1) such that 
the corresponding frequency response 

//(co)=//(z)U*> 

provides a good approximation to a desired response Hj{ to), i.e. 




(9.2) 


In order to be practically realizable, the resulting system H(z) must be stable and causal, which imposes 
strict constraints on the possible values of a k . Additionally, a linear phase requirement for H(z) may be 
desirable in certain applications, which further restrict the system parameters. 

It is important to realize that the use of an approximation in (9.2) is a fundamental necessity in practical 
filter design. In the vast majority of problems, the desired response H e /( go) does not correspond to a stable 
and causal system and cannot even be expressed exactly as a rational system function. 

To illustrate this point, recall from Chapter 2 that stability of H{z) implies that H( to) is continuous, while 
ideal frequency selective filters H^( go) all have discontinuities. 


Example 9.1: Ideal Low-Pass Filter 

► The ideal low-pass filter has already been studied in Example 3.3. The corresponding frequency re¬ 
sponse, denoted /T/(<n) in the present context, is illustrated in Figure 9.1. It is not continuous, and the 
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corresponding impulse response 

CD c . CD c n 

hd[n\ = —sinc(-) (9.3) 

7i n 

is easily seen to be non-causal. Furthermore, it is possible to show that 

£IM”]I = °° 

n 

so that the ideal low-pass filter is unstable. ◄ 



Fig. 9.1 Ideal Low-Pass filter specification in frequency. 


9.1.2 Specification of H ( i{(o) 

For frequency selective filters, the magnitude of the desired response is usually specified in terms of the 
tolerable 

• pass-band distortion, 

• stop-band attenuation, and 

• width of transition band. 

An additional requirement of linear phase (GLP) may be specified. 

For other types of filters, such as all-pass equalizers and Hilbert transformers, phase specifications play a 
more important role. 

Typical low-pass specifications: A typical graphical summary of design specifications for a discrete-time 
low-pass filter is shown in Figure 9.2. The parameters are: 

• [0, co p ] = pass-band 

• [ to s , it] = stop-band 

• Sco = <o 4 —co p = width of transition band. 

• Si = pass-band ripple. Often expressed in dB via 201og 10 (l + Si). 

• 82 = stop-band ripple. Usually expressed in dB as 201 og 10 ( 82 ). 
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1 + s { 

1 -S, 





> 

co 


Fig. 9.2 Typical template summarizing the design specifications for a low-pass filter. 


The phase response in the pass-band may also be specified, for example by an additional GLP requirement. 

Typically, a reduction of Si, 82 and/or Sen leads to an increase in the required HR filter order /V or FIR filter 
length M, and thus greater implementation costs. Ideally, one would to use the minimum value of N and/or 
M for which the filter specifications can be met. 


9.1.3 FIR or HR, That Is The Question 

The choice between HR and FIR is usually based on a consideration of the phase requirements. Recall that 
a LTI system is GLP iff 

H(<q) =A((o)e- j{aa> ~® (9.4) 

where the amplitude function A (co) and the phase parameters a and [3 are real valued. 

Below, we explain how the GLP requirement influences the choice between FIR and HR. In essence, only 
FIR filter can be at the same time stable, causal and GLP. This is a consequence of the following theorem, 
stated without proof. 


Theorem: A stable LTI system H(z) with real impulse response, i.e. h[n\ E M, is GLP if and only if 


H(z) = ez~ 2a H(l/z) 


(9.5) 


where a E M and £ E {— 1,+1}. 


PZ symmetry: 

• Suppose that p is a pole of H(z) with 0< \p\ < 1 . 

• According to above theorem, if H{z) is GLP, then: 

//(1/p) =ep 2a H(p) =00 (9.6) 

which shows that 1/p is also a pole of H(z). 
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• Note that p* and 1 /p* are also poles of H(z) under the assumption of a real system. 

• The above symmetry also apply to the zeros of H(z) 

• The situation is illustrated in the PZ diagram below: 



Fig. 9.3 PZ locations of HR real-coefficient GLP systems: a pole always appears with its 
inverse (and complex conjugate). 


Conclusion: If H(z) is stable and GLP, to any non-trivial pole p inside the unit circle corresponds a pole 
1 jp outside the unit circle, so that H(z) cannot have a causal impulse response. In other words, only FIR 
filter can be at the same time stable, causal and GLP. 


Basic design principle: 

• If GLP is essential ==> FIR 

• If not =>■ HR usually preferable (can meet specifications with lower complexity) 


9.2 Design of HR filters 


Mathematical setting: We analyze in this section common techniques used to design HR filters. Of 
course, we only concentrate on filters with rational system functions, as 


H(-\ - - b ° +b ' z 1 + ... + b M Z M 

A(z) l + a\z~ l + ...+a N z~ N 


(9.7) 


for which practical computational realizations are available. 

For H(z) in (9.7) to be the system function of a causal & stable LTI system, all its poles have to be inside 
the unit circle (U.C.). 
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Design problem: The goal is to find filter parameters N and {/?/ { } such that H( go) ~ Hd((o) to 
some desired degree of accuracy. 


Transformation methods: There exists a huge literature on analog filter design. In particular, several 
effective approaches do exist for the design of analog HR filters, say H a (D.). The transformation methods 
try to take advantage of this literature in the following way: 

(1) Map DT specifications Hj( to) into analog specifications H [U j(Q.) via proper transformation; 

(2) Design analog filter H a (£l) that meets the specifications; 

(3) Map H a (Q.) back into H (co) via a proper inverse transformation. 

9.2.1 Review of analog filtering concepts 

LTI system: For an analog LTI system, the input/ouput characteristic in the time domain takes the form 
of a convolution integral: 

/ oo 

h a (u)x a (t — u)du (9.8) 

-oo 

where h a (t) is the impulse response of the system. Note here that t E M, 


System function: The system function of an analog LTI system is defined as the Laplace transform of its 
impulse response, i.e. 

/ oo 

h a (t)e~ st dt (9.9) 

-oo 

where L denotes the Laplace operator. The complex variable s is often expressed in terms of its real and 
imaginary parts as 

5 = 0 + j Q 

The set of all s E C where integral (9.9) converges absolutely defines the region of convergence (ROC) of 
the Laplace transform. A complete specification of the system funciton H a {s) must include a description of 
the ROC. 

In the 5-domain, the convolution integral (9.8) is equivalent to 

Y a (s) = H a (s)X a (s) (9.10) 

where Y a (s ) and X a (s) are the Laplace transforms of y a (t) and x a {t). 


Causality and stability: An analog LTI system is causal iff the ROC of its system function H a (s) is a 
right-half plane, that is 

ROC : 91(5) > 0 O 

An analog LTI system is stable iff the ROC of H a (s) contains the j£l axis: 

jQ. E ROC for all O E M 

The ROC of a stable and causal analog LTI system is illustrated in the Fig. 9.4. 
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x 


x 


A lm(s) 



ROC 




► 

Re(s) 


Fig. 9.4 Illustration of a typical ROC for a causal and stable analog LTI system. 

Frequency response: The frequency response of an LTI system is defined as the Fourier transform of its 
impulse response, or equivalently 1 

H a (£l) = H a (s)\ s=j Q (9.11) 

In the case an analog system with a real impulse response, we have 

|//„(Li)| 2 = H a (s)H a (-s)\ s =jn (9.12) 


Rational systems: As the name indicates, a rational system is characterized by a system function that is a 
rational function of s, typically: 


_ Po + Pa h -h P m s m 

1T cci^ T • • • T ccj\/s n 


(9.13) 


The corresponding input-output relationship in the time-domain is an ordinary differential equation of order 
N, that is: 


N jk M ,k 

y a (t ) = - £ a k-7-^y a {t) + £ P k-j^X a (t) 

k= 1 az k=0 al 


(9.14) 


9.2.2 Basic analog filter types 

In this course, we will basically consider only three main types of analog filters in the design of discrete¬ 
time HR filters via transformation methods. These three types will have increasing “complexity” in the sense 
of being able to design a filter just with a paper and pencil, but will also have an increasing efficiency in 
meeting a prescribed set of specifications with a limited number of filter parameters. 


Butterworth filters: Butterworth filters all have a common shape of squared magnitude response: 


\H a (Cl)\ 2 = 




n2 A?’ 


'Note the abuse of notation here: we should formally write H a (jO.) but usually drop the j for simplicity. 


(9.15) 
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where Q. c is the cut-off frequency (-3dB) and N is the filter order. Figure 9.5 illustrates the frequency 
response of several Butterworth filters of different orders. Note that \H a (Q.) smoothly decays from a maxi- 



Radian frequency Q. 

Fig. 9.5 Magnitude response of several Butterworth filters of different orders. (£2 C = 
10 rad / s) 


mum of 1 at Q. = 0 to 0 at Q. = °o. These filters are simple to compute, and can be easily manipulated without 
the help of a computer. The corresponding transfer function is given by: 


H a (s) = 




\N 


(s - s 0 )(s - Si) ■ ■ ■ {s - S N -l) 

The poles Sk are uniformly spaced on a circle of radius Cl c in the .v-placc, as given by: 

s k = £2 c exp(;[^ + (*+ k = 0,l,...,N-l. 


(9.16) 


(9.17) 


Example 9.2: 

► 


◄ 


Chebyshev filters: Chebyshev filters come in two flavours: type I and II. The main difference with the 
Butterworth filters is that they are not monotonic in their magnitude response: Type I Chebyshev filters have 
ripple in their passband, while Type II Chebyshev filters have ripple in their stopband. The table below 
summarizes the features of the two types of Chebyshev filters, and also gives an expression of their squared 
magnitude response: 



Type I 

Type II 

Def 


lu fr>'i|2 I 

1 c ( l+e 2 T${Q/Q c ) 

1 ci ““ j| 1 +(e’72(fi c /n))- 1 

Passband 

Ripple from 1 to \J \/1 + e 2 

Monotonic 

Stopband 

Monotonic 

Ripple v / l/(l + l/ £2 ) 
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In the above table, the function Tn(-) represents the /V-th order Chebyshev polynomial. The details of the 
derivation of these polynomials is beyond the scope of this text, but can be obtained from any standard 
textbook, e is a variable that determines the amount of ripple, and £l c is a cut-off frequency. Figure 9.6 
illustrates the frequency response of these filters. Chebyshev filters become more complicated to deal with 
than Butterworth filters, mainly because of the presence of the Chebyshev polynomials in their definition, 
and often they will be designed through an ad-hoc computer program. 


Chebyshev Type I Chebyshev Type II 



Fig. 9.6 Illustration of the magnitude response of analog Chebyshev filters of different orders 
N: (a) Type I, (b) Type II. 


Elliptic filters: Elliptic filters are based on elliptic functions. Their exact expression is beyond the scope 
of this text, but they can be easily generated by an appropriate computer program. Figure 9.7 illustrates the 
magnitude response of these filters. One can see that they ripple both in the passband and stop band. They 
usually require a lower order than Chebyshev and Butterworth filters to meet the same specifications. 


9.2.3 Impulse invariance method 


Principle: In this method of transformation, the impulse response of the discrete-time filter H(z) is ob¬ 
tained by sampling the impulse response of a properly designed analog filter H a (s ): 

h[n] = h a (nT s ) (9.18) 


where T s is an appropriate sampling period. From sampling theory in Chapter 7, we know that 


u t \ 1 V" -1 ® 2nk 

1 S L *S 


(9.19) 


If there is no significant aliasing, then 


tf(C0) = —H a &), |G>|<7C. 

*S *S 


(9.20) 
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Elliptic filters 



Fig. 9.7 Magnitude Response of low-pass elliptic filters of different orders. 



Fig. 9.8 Illustration of aliasing with impulse invariance method. 


Thus in the absence of aliasing, the frequency response of the DT filter is the same (up to a scale factor) as 
the frequency response of the CT filter. 

Clearly, this method works only if |T4(£2)| becomes very small for |£2| > %/T s , so that aliasing can be 
neglected. Thus, it can be used to design low-pass and band-pass filters, but not high-pass filters. 

In the absence of additional specifications on H a {D), the parameter T s can be set to 1: it is a fictitious 
sampling period, that does not correspond to any physical sampling. 


Design steps: 


(1) Given the desired specifications o) for the DT filter, find the corresponding specifications for the 
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analog filter, H cc i(D.)'. 

H ad {D) = T s H d (to)|o>=£ 2 Tj, |fl| < £ (9.21) 

For example, Figure 9.9 illustrates the conversion of specifications from DT to CT for the case of a 
low-pass design. 



Fig. 9.9 Conversion of specifications from discrete-time to continuous-time for a design by 
impulse invariance. 


(2) Design an analog HR filter H a (Cl) that meets the desired analog specifications H ad (Q.): 

- Chose the filter type (Butterworth, elliptic, etc.) 

- Select the filter parameters (filter order N, poles Sk, etc.) 

(3) Transform the analog filter H a (Q.) into a discrete-time filter H(z) via time-domain sampling of the 
impulse response: 


Kit) 

= L~ l {H a {Q.)} 


h[n\ 

= K(nT s ) 


H(z) 

= Z{h[n}} 

(9.22) 

In practice, it is not necessary to find h a {t), 

as explained below. 



Simplification of step (3): Assume H a (s) has only first order poles (e.g. Butterworth filters). By perform¬ 
ing a partial fraction expansion of H a (s), one gets: 


H a (s) 


N 


I 


Ak 

s-s k 


The inverse Laplace transform of H a (s) is given by 

N 

h a (t) = Y, A ke Skt u c {t) 

k= l 


where u a (t) is the analog unit step function, i.e. 

u c {t) 


1 , t > 0 
0 , t < 0 


(9.23) 


(9.24) 


(9-25) 
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Now, if we sample h a (t) in (9.24) uniformly at times t = nT s , we obtain 


N N 


h[n ] = Y, A ke SknTs u[n\ = £ A k (e SkT ’) n u[n]. 


(9.26) 



The Z-transform of h[n\ is immediately obtained as: 


N A, 

H(z) = V -—r where pk = e SkTs 

£i l ~Pkt 


(9.27) 


This clearly shows that there is no need to go through the actual steps of inverse Laplace transform, sampling 
and Z-transform: one only needs the partial fraction expansion coefficients Ak and the poles from (9.23) 


and plug them into (9.27). 

Remarks: The order of the filter is preserved by impulse invariance, that is: if H a (s) is of order N, then 
so is H(z )■ The correspondence between poles in discrete-time and continuous-time is given by: 


sic is a pole of H a (s ) => pu = e SkTs is a pole of H[z). 


As a result, stability and causality of the analog filter are preserved by the transformation. Indeed, if H a (s) 
is stable and causal, then the real part of all its poles s * will be negative: 

Sk — €>k T jQ-ki ®k T 0. 

The corresponding poles pk in discrete-time can be written as 

p k = e akTs e J QkTs 

Therefore, 

\pk\ =e CkTs < 1, 

which ensures causality and stability of the discrete-time filter H(z)- 

Example 9.3: 

► Apply the impulse invariance method to an appropriate analog Butterworth filter in order to design a 
low-pass digital filter H(z) that meets the specifications outlined in Figure 9.10. 


• Step 1: We set the sampling period to T s = 1 (i.e. Q. = co), so that the desired specifications for the 
analog filter are the same as stated in Figure 9.10, or equivalently: 


1 - 8i < \H ad (Cl)\ < 1 for 0<|£2|<fti = .257: 
|/Mfl)|<52, for = -4 jc < \C1\ < - 


(9.28) 

(9.29) 


• Step 2: We need to design an analog Butterworth filter H c (£f) that meets the above specifications. 
For the Butterworth family, we have 



(9.30) 


Thus, we need to determine the filter order N and the cut-off frequency fT so that the above 
inequalities are satisfied. This is explained below: 
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Fig. 9.10 Specifications for the low-pass design of example 9.3 


- We first try to an find analog Butterworth filter that meets the band edge conditions exactly, 
that is: 


1^(1201 = 1 - 5 , 


J_ ^ 

i + (i2,/a-) 2iV = (1 ~ 8l) ~ 

(12,/HO 2 *' = (7~802 ~ 1 = a 


(9.31) 


|flfl(ft2)|=8 2 


i + (n 2 /i 2 c ) 2 * (52)2 

(£k/toc) w = ^ - 1 = P 


Combining these two equations, we obtain: 


pi/QO 2 " a 

v^2/nj “ p 


=>2Aln(12,/12 2 ) =ln(oc/p) 


1 ln(a/p) 

2 ln(12,/12i) 


12.16 


Since N must be an integer, we take IV = 13. 

- The value of 12 f is then determined by requiring that: 


1 / 0(1201 = 1 - 5 , 


1 + (12, / 12 c ) 26 (1 5l) 

12 c = 0.856 


(9.32) 


(9.33) 


With this choice, the specifications are met exactly at 12, and are exceeded at 12 2 , which 
contribute to minimize aliasing effects. 

- An analog Butterworth filter meeting the desired specifications is obtained as follows: 


n ( i (a)13 

CW (s-So)(«-*l)-(*-St2) 

(9.34) 

s k = 12 c .e- /[ 2 +(i:+ 2 ) n] k = 0,1, •■■,12 

(9.35) 
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• Step 3: The desired digital filter is finally obtained by applying the impulse invariance method: 
- Partial fraction expansion: 


H c (s) = 


Ap 

s-so 


Direct mapping to the z-domain: 

H{z) = 


Ap 


\-ppz 1 


An 

s-sn 

An 

1-pnz -1 


(9.36) 


(9.37) 


with 

p k = e s \ k = 0,1,..., 12 (9.38) 

Table 9.1 lists the values of the poles Sk , the partial fraction expansion coefficients Ak and the 
discrete-time poles pi- for this example. 


k 

Sk 

Ak 

Pk 

0 

-0.1031 + 0.8493; 

-0.0286 + 0.2356; 

+0.5958 + 0.6773; 

1 

-0.3034 + 0.8000; 

-1.8273-0.6930; 

+0.5144 + 0.5296; 

2 

-0.4860 + 0.7041 j 

+4.5041-6.5254; 

+0.4688 + 0.3982; 

3 

—0.6404 + 0.5674; 

+ 13.8638 + 15.6490; 

+0.4445 + 0.2833; 

4 

—0.7576 + 0.3976 j 

—35.2718 +18.5121 j 

+0.4322 + 0.1815; 

5 

-0.8307 + 0.2048; 

-13.8110-56.0335; 

+0.4266 + 0.0886; 

6 

-0.8556 + 0.0000; 

+65.1416 + 0.0000; 

+0.4250 + 0.0000; 

7 

-0.8307-0.2048; 

-13.8110 + 56.0335; 

+0.4266-0.0886; 

8 

-0.7576-0.3976; 

—35.2718 — 18.5121 j 

+0.4322-0.1815; 

9 

0.6404-0.5674; 

+ 13.8638- 15.6490; 

+0.4445-0.2833; 

10 

-0.4860-0.7041; 

+4.5041 + 6.5254; 

+0.4688-0.3982; 

11 

-0.3034-0.8000; 

-1.8273 + 0.6930; 

+0.5144-0.5296; 

12 

-0.1031-0.8493; 

-0.0286-0.2356; 

+0.5958-0.6773; 


Table 9.1 Values of the coefficients used in example 9.3. 


- At this point, it is easy to manipulate the function H(z ) to derive various filter realizations 
(e.g. direct forms, cascade form, etc.). Figure 9.11 illustrates the magnitude response of the 
filter so designed. 


◄ 


9.2.4 Bilinear transformation 

Principle: Suppose we are given an analog HR filter H a (s). A corresponding digital filter H(z) can be 
obtained by applying the bilinear transformation (BT), defined as follows: 

H(z)=H a (s) at s = <K z )A a Qz£_y (9.39) 

where a is a scaling parameter introduced for flexibility. In the absence of other requirements, it is often set 
to 1. 
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Fig. 9.11 Magnitude Response of the design of example 9.3. 


Example 9.4: 


► Consider the analog filter 

H a (s) = — b> 0 (9.40) 

s + b 

Up to a scaling factor, this is a Butterworth filter of order A' = 1 with cut-off frequency £2 e = b. This filter 
has only one simple pole at s = —b in the ,v-plane; it is stable and causal with a low-pass behavior. The 
corresponding PZ diagram in the ,v-plane and magnitude response versus £2 are illustrated Figure 9.14 
(left) and Figure 9.13 (top), respectively. Note the -3dB point at £2 = b in this example. 

Applying the BT with a = 1, we obtain 


m 


i 

l-Z - 1 

T+F 1 

1 


+ b 

1+F 1 




(9.41) 


The resulting digital filter has one simple pole at 


— 4 
1 +b 


inside the U.C.; it is stable and causal with a low-pass behavior The corresponding PZ diagram in the 
Z-plane and magnitude response versus to are illustrated Figure 9.12 (right) and Figure 9.13 (bottom), 
respectively. 


◄ 


Properties of bilinear transformation: 

5 = 4>(z) = a 


The inverse mapping is given by: 

, , , i, , l+s/a 

= * 1 = Y^sja 


(9.42) 
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Fig. 9.12 PZ diagrams of the filters used in example 9.4. Left: analog filter (b = .5); Right: 
discrete-time filter. 


IH.(£2)I 



Fig. 9.13 Magnitude responses of the filters used in example 9.4. Top: analog filter (b = .5); 
Bottom: discrete-time filter. 
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It can be verified from (9.42) that the BT maps the left-half part of the 5-plane into the interior of the unit 
circle in the z-plane, that is: 

s = c + jD with a < 0 <=> |z| < 1 (9.43) 

If H a (s) has a pole at s k , then H{z) has a corresponding pole at 

Pk = <\>~\s k ) (9.44) 

Therefore, if H a (s) is stable and causal, so is H(z). This is illustrated in Figure 9.14. 



Fig. 9.14 Inverse BT mapping from 5-plane to z-plane operated. 


The BT uniquely maps the jD axis in the 5-plane onto the unit circle in the z-plane (i.e. 1 -to-1 mapping): 


s = jD. 


■z = 


1 T jD/rx 
1 - jD /a 


A 

A* 


(9.45) 


where A = 1 + jD / a. Clearly 


|z| = |A|/|A*| = l 


(9.46) 


The relationship between the physical frequency D and the normalized frequency to can be further developed 
as follows: 

s = jQ.^z = e ja (9.47) 

where to = LA — LA* = 2LA, that is 

co = 2tan~ 1 (p (9.48) 

or equivalently 

fl = atan(^) (9.49) 

This is illustrated in Figure 9.15, from which it is clear that the mapping between D and to is non-linear. 
In particular, the semi-infinite D axis [0,°o) is compressed into the finite to interval [0,7l]. This non-linear 
compression, also called frequency warping, particularly affects the high-frequencies. It is important to 
understand this effect and take it into account when designing a filter. 
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Fig. 9.15 Illustration of the frequency mapping operated by the bilinear transformation. 

Design steps: 

(1) Given desired specifications H d ( co) for the DT filter, find the corresponding specifications for the 
analog filter, H ad (D.): 

H ad (Q.) = H d ( co) Q. = a tan(“) (9.50) 

(2) Design an analog HR filter H a (Q.) that meets these spec. 

(3) Apply the BT to obtain the desired digital filter: 

H(z) = H a (s) at t = (9.51) 


Example 9.5: 

► Apply the BT to an appropriate analog Butterworth filter in order to design a low-pass digital filter H(z) 
that meets the specifications described in Figure 9.10. 

• Step 1: In this example, we set the parameter a to 1. The desired specifications for the analog 
filter are similar to the above except for the band-edge frequencies. That is 

1 — 5i < \H ad (Q.)\ = 1 for 0<|D|<D, (9.52) 

\H ad (Cl)\<5 2 , for fl 2 =<|fl|<~ (9.53) 

where now 


,0,1 

t ,C0i 

= tan( — )p 
1 2 ’ 

s 0.414214 


02 

= m( t ) S 

a 0.72654 

(9.54) 


• Step 2: We need to design an analog Butterworth filter H a (£l) that meets the above specifications. 

- Proceeding exactly as we did in Example 9.3, we find that the required order of the Butter¬ 
worth filter is N = 11 (or precisely N > 10.1756). 

- Since aliasing is not an issue with the BT method, either one of the stop-band edge specifica¬ 
tions may be used to determine the cut-off frequency Cl c . For example: 

\H a (n 2 )\ = - , \ = S 2 => Oc = 0.478019 (9.55) 

1 + (n 2 /n c ) 22 
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The desired analog Butterworth filter is specified as 

( a -) 11 


H c (s) = 


(s - so)(s - Si) ■ ■ ■ (s - Sio) 

= '] A: = 0,1,..., 10 

• Step 3: The desired digital filter is finally obtained by applying the bilinear transformation: 

( a ) 11 


(9.56) 

(9.57) 


H(z) = 


bo + b\z~ l H- b n z~ n 


1+fliZ - anz 11 

The corresponding values of the coefficients b, and a, are shown in Table 9.2. 


(9.58) 


k 

bk 

a k 

0 

+0.0003 

+26.5231 

1 

+0.0033 

-125.8049 

2 

+0.0164 

+296.9188 

3 

+0.0491 

-446.8507 

4 

+0.0983 

+470.4378 

5 

+0.1376 

-360.6849 

6 

+0.1376 

+204.3014 

7 

+0.0983 

-85.1157 

8 

+0.0491 

+25.4732 

9 

+0.0164 

-5.2012 

10 

+0.0033 

+0.6506 

11 

+0.0003 

-0.0377 


Table 9.2 Coefficients of the filter designed in example 9.5. 


9.3 Design of FIR filters 

Design problem: The general FIR system function is obtained by setting A(z) = 1 in (9.1), that is 

M 

H(z) = B(z)='£b kZ - k 
k =o 

The relationship between coefficients bk and the filter impulse response is 

, r I _/ K 0 <n<M 
111 \ 0 otherwise 


(9.59) 


(9.60) 
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In the context of FIR filter design, it is convenient to work out the analysis directly in terms of h \n\. Accord¬ 
ingly, we shall express H(z) as: 


H(z) = h[0] + h[l]z~ l + • • -h[M] z - M 


(9.61) 


It is further assumed throughout that h[ 0] / 0 and h\M] ^ 0. 

The design problem then consists in finding the degree M and the filter coefficients {h \k \} in (9.61) such that 
the resulting frequency response approximates a desired response, H d ( co), that is 


H((0)~H d ((0) 


In the design of FIR filters: 

• stability is not an issue since FIR filters are always stable; 

• considerable emphasis is usually given to the issue of linear phase (one of the main reasons for select¬ 
ing an FIR filter instead of an HR filter). 

In this section, we will review a classification of FIR GLP systems, then explain the principles of design by 
windowing and design by minmax optimization. 

9.3.1 Classification of GLP FIR filters 

We recall the following important definition and result from Section 8.3.3: 

Definition: A discrete-time LTI system is GLP iff 


//(co) = A ((£>)e ^ 


(9-62) 


where the A(co), a and [3 are real valued. 


Property: A real FIR filter with system function H(z) as in (9.61) is GLP if and only if 


h[k]=eh[M-k] k = 0,1, ...,M 

where e is either equal to +1 or — 1. 


(9.63) 


Example 9.6: 


► Consider the FIR filter with impulse response given by 

h[n] = {1,0,-1,0,1,0,-1} 


(9.64) 


Note that here, h[n] = h\M — n\ with M = 6 and 8=1. Let us verify that the frequency response //(to) 
effectively satisfy the condition (9.62): 


H (e j(0 ) = 1 - e~ ; “ 2 + e~ ja4 - e~ ja6 


H(e j(0 ) 


= e~ j(o3 (e ja3 - e ja + e~ ja - e ~ jm3 ) 
= 2Je“ ;0)3 (sin(3co) — sin(co)) 

= A(co)e-^“ c °-P ) 
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where we define the real-valued quantities 


371 

A(co) = 2sin(co) — 2sin(3co), a=3, p= — 


The frequency response is illustrated in Figure 9.16. 




-71/2 


ji/2 


frequency co 

Fig. 9.16 Example frequency response of a generalized linear-phase filter. 


Classification of GLP FIR filters: Depending on the values of the parameters £ and M, we distinguish 4 
types of FIR GLP filters: 



M even 

M odd 

e = +l 

Type I 

Type II 

£ = -1 

Type III 

Type IV 


A good understanding of the basic frequency characteristics of these four filter Types is important for practi¬ 
cal FIR filter design. Indeed, not all filter Types, as listed above, can be used say for the design of a low-pass 
or high-pass filter. This issue is further explored below: 


• Type I (e = +1, M even): 

H( co) = e~ jaM ^ 2 {^akCos((ok)} 

k 

• Type II (s = +1 ,M odd): 

H( ( o)=e-^/ 2 {^ k cos^k- 1 -))} 

k z 


(9.65) 


(9.66) 
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- H( co) = 0 at co = 71 

- Cannot be used to design high-pass filter 

• Type III (£ = — 1, M even): 

//(co) = je~ Ja M/2 {J>sin(co/)} (9.67) 

k 

- //(co) = 0 at co = 0 and at co = Jt 

- Cannot be used to design either low-pass or high-pass filter 

• Type IV (£ = — 1, M odd): 

//(co) = (9.68) 

k z 

- //(co) = 0 at co = 0 

- Cannot be used to design low-pass filter 


9.3.2 Design of FIR filter via windowing 


Basic principle: Desired frequency responses //./(co) usually correspond to non-causal and non-stable 
filters, with infinite impulse responses h c i[n\ extending from — °o to °o. Since these filters have finite energy, 
it can be shown that 


\h[n] 


0 as n 


(9.69) 


Based on these considerations, it is tempting to approximate h t /[n] by an FIR response g\n] in the following 
way: 

_J M»]> \ n \< K (9 70) 


(9.71) 


P\tl\ — 

1 1 '0, otherwise 

The FIR g[n] can then be made causal by a simple shift, i.e. 

h[n] = g[n—K] 


Example 9.7: 

► Consider an ideal low-pass filter: 


///(co) 


1, |co| < C0 c = Jt/2 


0, CD c < |co| < 7t 
The corresponding impulse response is given by 

. nC0 c 1 . n 

hd[n\ = —smc( —) = -sinc(-) 

71 71 2 2 

These desired characteristics are outlined in Figure 9.17. 

Assuming a value of K = 5, the truncated FIR filter g[n] is obtained as 


sM = 


hj [n], —5<n<5 


0, 


otherwise 


© B. Champagne & F. Labeau 


Compiled November 5, 2004 




9.3 Design of FIR filters 


187 


Finally, a causal FIR with M = 2 K is obtained from g [n] via a simple shift: 

h[n\ =g[n- 5] 

These two impulse responses are illustrated in Figure 9.18. The frequency response of the resulting 
filter, G(co) is illustrated in Figure 9.19. ◄ 



Fig. 9.17 Desired (a) frequency response and (b) impulse response for example 9.7 



Fig. 9.18 (a) Windowed impulse response g[n) and (b) shifted windowed impulse response 
h[n) for example 9.7 


Observation: The above design steps (9.70)- (9.71) can be combined into a single equation 


where 


According to (9.72), the desired impulse response h d [n\ is first shifted by K = M/2 and then multiplied by 

W R [ll\. 


M 

h[n] = w R [n\ h d [n - —} 

(9.72) 

r i a [ 1, 0 <n<M 

Wr n | 0, otherwise 

(9.73) 


Remarks on the window: The function wr[h] is called a rectangular window. More generally, in the 
context of FIR filter design, we define a window as a DT function w[n] such that 


( w[M — n\> 0, 0 <n<M 
\ 0, otherwise 


(9.74) 


As we explain later, several windows have been developed and analysed for the purpose of FIR filter design. 
These windows usually lead to better properties of the designed filter. 
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Fig. 9.19 Example frequency response of an FIR filter designed by windowing (refer to ex¬ 
ample 9.7). 

Remarks on time shift: Note that the same magnitude response |//(co) | would be obtained if the shift 
operation in (9.72) was omitted. In practice, the shift operation is used to center hd[n] within the window 
interval 0 < n < M, so that symmetries originally present in h d [n] with respect to n = 0 translate into corre¬ 
sponding symmetries of h[n\ with respect to n = K = M/2 after windowing. In this way, a desired response 
H d (co) with a GLP property, that is 


h d [n] = th d [—n] n e Z 

is mapped into an FIR filter //(to) with GLP, i.e. 

(9.75) 

h[n\=w [n] hd [n — K] =zh[M — n\ 

(9.76) 

as required in most applications of FIR filtering. 

The above approach leads to an FIR filter with M = 2K even. To accommodate the case M odd, or equiva¬ 
lently K = M/2 non-integer, the following definition of a non-integer shift is used: 

h d [n-K] = [* m co)e-* K }e^d<o 

271 J —n 

(9.77) 

Design steps: To approximate a desired frequency response Hj( to) 

• Step (1): Choose window w[n] and related parameters (including M). 

• Step (2): Compute the shifted version of desired impulse response: 


h d [n-K\ = ^ r {H d ((0)e-J aK }eJ mn d(0 

271 J—n 

(9.78) 


where K = M/2. 
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• Step(3): Apply the selected window w[n)\ 

h[n\ = w[n\h c i[n — K\ (9.79) 


Remarks: Several standard window functions exist that can be used in this method (e.g. Bartlet, Hanning, 
etc.). 

For many of these windows, empirical formulas have been developed that can be used as guidelines in the 
choice of a suitable M. 

To select the proper window, it is important to understand the effects of and trade-offs involved in the 
windowing operation (9.79) in the frequency domain. 


Spectral effects of windowing: Consider the window design formula (9.79), with shift parameter K set 
to zero in order to simplify the discussion, i.e. 

h[n] = w[n\hd[n] (9.80) 


Taking the DTFT on both sides, we obtain: 

HU a) = ^~ T H d (B)wu o - ewe 

271 J-n 


(9.81) 


W'(co) = £ w[n\e- jan (9.82) 

n —0 

is the DTFT of the window function w[n]. 

Thus, application of the window function w[n] in the time-domain is equivalent to periodic convolution with 
W (to) in the frequency domain. 

For an infinitely long rectangular window, i.e. w[n] = 1 for all n 6 Z, we have W (to) = 2jt£/. 5 c (co — 2nk) 
with the result that H( co) = H ( i{ go). In this limiting case, windowing does not introduce any distortion in the 
desired frequency response. 

For a practical finite-length window, i.e. time limited to 0 < n < M, the DTFT W (co) is spread around co = 0 
and the convolution (9.81) leads to smearing of the desired response H c i( co). 


Spectral characteristics of the rectangular window: For the rectangular window wr [n] = 1,0 <n <M, 
we find: 

M 

Wr(co) = £ e- jan (9.83) 

n —0 

-jwM/2 ( sin(co(M+ l)/2) \ 

' V sin(co/2) ) 

The corresponding magnitude spectrum is illustrated below for the M = 8: 

|Wr(oo)| is characterized by a main lobe, centered at co = 0, and secondary lobes of lower amplitudes, also 
called sidelobes: For wr[ n ], we have: 
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- mainlobe width: Ago,,, = 

- peak side lobe level: a = — 13dB (independent of M) 


Typical LP design with window method: Figure 9.20 illustrates the typical look of the magnitude re¬ 
sponse of a low-pass FIR filter designed by windowing. Characteristic features include: 

• Ripples in |//(co)| due to the window sidelobes in the convolution (Gibb’s phenomenon); 

• 8, which denotes the peak approximation error in [0, G0 /; ] U [to s , 7t]. 

8 , strongly depends on the peak sidelobe level a of the window. 

• Width of transition band: Ago = co v — (O p < Ago,,,. Note that Ago / 0 due to main lobe width of the 
window; 

• | //(go) | / 0 in stopband =» leakage (also an effect of sidelobes) 

For a rectangular window: 

• 8 « -21dB (independent of M) 

• Aca«g 

To reduce 8, one must use an other type of window (Kaiser, Hanning, Hamming,...). These other windows 
have wider main lobes (resulting in wider transition bands) but in general lower sidelobes (resulting in lower 
ripple). 
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co 


Fig. 9.20 Typical magnitude response of an FIR filter designed by windowing. 

9.3.3 Overview of some standard windows 

Windowing trade-offs: Ideally, one would like to use a window function w[n\ that has finite duration in 
the time-domain and such that its DTFT W(co) is perfectly concentrated at co = 0. This way, the application 
of w[n] to a signal x[n] in the time-domain would not not introduce spectral smearing, as predicted by (9.81). 
Unfortunately, such a window function does not exist: it is a fundamental property of the DTFT that a signal 
with finite duration has a spectrum that extends from — °o to +°o in the co-domain. 

For a fixed value of the window length M, it is possible however to vary the window coefficients w\n\ so as 
to trade-off main-lobe width for sidelobe level attenuation. In particular, it can be observed that by tapering 
off the values of the window coefficients near its extremity, it is possible to reduce sidelobe level a at the 
expense of increasing the width of the main lobe, Act),,,. This is illustrated in Figure 9.21: We refer to this 
phenomenon as the fundamental windowing trade-off. 


Bartlett or triangular window: 


w[n\ = (1 


2 n 
~M 


l\)w R [n\ 


Hanning Window: 


w[n] = ^(1-cos 2 Jf) w R[n] 


Hamming Window: 


w[n] = (0.54 — 0.46 cos 



w R [n] 


(9.84) 


(9.85) 


(9.86) 


Blackman Window: 


w[n] = (0.42 — 0.5 cos 


2n n 
M 


+ 0.08 cos 


47t n 
M 


w R [n\ 


(9.87) 
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i 

0.8 

-d 0.6 
Q. 

I 0.4 
0.2 


rectangular window 


triangular window 


OM- 


5 10 

time index n 


15 





Fig. 9.21 Illustration of the fundamental windowing tradeoff. 

A generic form for the Hanning, Hamming and Blackman windows is 

Jirn Ann 

win ] = (A + Bcos-bCcos-) wr [n] 

L J M M 1 

where coefficients A, B and C determine the window type. 

Kaiser window family: 

w[n] = JM r ° { ^ ] ~ ( ^~ V)2)wR[n] 


(9.88) 


where 7 0 (.) denotes the Bessel function of the 1st kind and P > 0 is an adjustable design parameter. By 
varying a, it is possible to trade-off sidelobe level a for mainlobe width: Aco,„: 

• P = 0 =>• rectangular window; 

• p > 0 => a | and Aco m |. 

The so-called Kaiser design formulae enable one to find necessary values of p and M to meet specific low- 
pass filter requirements. Let Aco denote the transition band of the filter i.e. 


AtO = CD S - OOp 

and let A denote the stopband attenuation in positive dB: 

A = — 201og 10 8 


(9.89) 


(9.90) 
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time index n 



—71 -71/2 0 71/2 71 




-71 -71/2 0 7t/2 71 

frequency to 


Fig. 9.22 


The following formulae 2 may be used to determine [3 and M: 

A - 8 

M ~ 2.285Aco 


(9.91) 


P = 


0.1102(A — 8.7) 

< 0.5842(A — 21)°- 4 + 0.07886(A —21) 
0.0 


A >50 
21 <A<50 
A < 21 


(9.92) 


2 J. F. Kaiser, ’’Nonrecursive digital filter design using the l a — sinh window function,” IEEE Int. Symp. on Circuits and Systems, 
April 1974, pp. 20-23. 
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Fig. 9.23 


9.3.4 Parks-McClellan method 

Introduction: Consider a Type-I GLP FIR filter (i.e., M even and £ = +1): 

//(co) = e- j(oM/2 A{c o) (9.93) 

L 

A (to) = 2L a ' tCOS (^ C0 ) (9.94) 

k =o 

where L = M/2, oto = h [L], and 04 = 2/z[L — k ] for k = 1,..., L . Let (co) be the desired frequency response 
(e.g., ideal low-pass filter), specified in terms of tolerances in frequency bands of interest. 

The Parks-McClellan (PM) method of FIR filter design attempts to find the best set of coefficients oq.’s, in 
the sense of minimizing the maximum weighted approximation error 

E( co) = W(co)[Zfc(a>) -A(co)] (9.95) 
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over the frequency bands of interest (e.g., passband and stopband). W(co) > 0 is a frequency weighting 
introduced to penalize differently the errors made in different bands. 

Specifying the desired response: The desired frequency response H c /( co) must be accompanied by a tol¬ 
erance scheme or template which specifies: 

- a set of disjoint frequency bands ®,- = [co,i, 00,2] 

- and for each band ®,-, a corresponding tolerance 8,-, so that 

\Hd(&) —A(co)| < Sj, cog®, ( 9.96) 

The frequency intervals between the bands ®, are called transition bands. The behavior of A(co) in the 
transition bands cannot be specified. 

Example 9.8: 

► For example, in the case of a low-pass design: 

- Passband: 'B\ = [0. CO,,] with tolerance 81, i.e. 

|1 — A(oo)| < 81, 0 < co < co p (9.97) 

- Stopband: 'lh = [co v , 71 with tolerance 82, i.e. 

|A(co) | < 82, co s < co < Jt (9.98) 


- Transition band: [co ;) . co v ] 

◄ 


The weighting mechanism: In the Parks-McClellan method, a weighting function VK(co) > 0 is applied 
to the approximation error H d ( co) — A(oo) prior to its optimization, resulting in a weighted error 

E( co) = W{( 0 )[H d (( 0 ) -A(co)]. (9.99) 

The purpose of W (co) is to scale the error H d ( co) — A(co), so that an error in a band ®, with a small tolerance 
8,- is penalized more than an error in a band Bj with a large tolerance 8/ > 8 ,-. This may be achieved by 
constructing W(co) in the following way: 


W'(co) = COG®, (9.100) 

8 i 

so that the cost W (co) is larger in those bands ®, where 8, is smaller. In this way, the non-uniform tolerance 
specifications 8, over the individual bands ®- translate into a single tolerance specification of over all the 
bands of interest, i.e. 

|£(co)|<l, CO G ®i U ® 2 U ... (9.101) 
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Properties of the min-max solution: In the Parks-McClellan method, one seeks an optimal filter H( to) 
that minimizes the maximum approximation error over the frequency bands of interest. Specifically, we 
seek 


where 


min { max |2s(go)| } 

(9.102) 

all (X£ ooe(B 

E( (D) = W(G0)[tfrf(G0) -A(go)] 

(9.103) 

L 

A (go) = V a/;Cos(kGo) 

(9.104) 

k= 0 

3 = «1 U 32 U ... 

(9.105) 


The optimal solution A 0 (co) satisfies the following properties: 

- Alternation theorem: A 0 (co) is the unique solution to the above min-max problem iff it has at least 
L + 2 alternations over 3. 


- That is, there must exist at least L + 2 frequency points co, E 3 such that coi < Ob < • ■ ■ < (Ol+2 and 
that 


E((SH) = —E((Oi+\) = ±Ej 


(9.106) 


F m ax = max |E(co)| 

COG® 


(9.107) 


- All interior points of 3 where A (go) has zero slope correspond to alternations. 
This is illustrated in Figure 9.24 for a low-pass design. 



Fig. 9.24 Illustration of the characteristics of a low-pass filter designed using the Parks- 
McClellan method. 


© B. Champagne & F. Labeau 


Compiled November 5, 2004 


















9.3 Design of FIR filters 


197 


Remarks: Because of the alternations, these filters are also called equiripple. Due to the use of the weight¬ 
ing function W(co), an equiripple behavior of E((o) over the band $ with amplitude E max translates into 
equiripple behavior of Ft c i( co) —A(co) with amplitude 8,7? max over band %. 


Observe that the initial specifications in (9.96) will be satisfied if the final error level E max < 1. One practical 
problem with the PM method is that the required value of M to achieve this is not known a priori. In other 
words, the method only ensures that the relative sizes of the desired tolerances are attained (i.e. the ratio 
5 ,-/ 5 /). 


In the case of a low-pass design, the following empirical formula can be used to obtain an initial guess for 


M: 


-lOlog^StSz)-^ 

2.324(00,. - C0p) 


(9.108) 


Although we have presented the method for Type-I GLP FIR filter, it is also applicable with appropriate 
modifications to Types II, III and IV GLP FIR filters. Generalizations to non-linear phase complex FIR 
filters also exist. 


Use of computer program: Computer programs are available for carrying out the minimax optimization 
numerically. They are based on the so-called Remez exchange algorithm. In the Matlab signal processing 
toolbox, the function remez can be used to design minimax GLP FIR filters of Type I, II, III and IV. The 
function cremez can be used to design complex valued FIR filters with arbitrary phase. 


Example 9.9: 


► Suppose we want to design a low-pass filter with 

- Passband: |1 —//(co)| < 8i = .05, 0 < co < co p = 0.47t 

- Stopband: |//(co)| <82 = -01, 0.67t = co, < to < n 


• Initial guess for M: 


— 101og 10 (0.05 x 0.01) — 13 
2.324(0.671 -0.471) 


• Set M = 11 and use the commands: 


10.24 


(9.109) 


F = 

[0,0.4,0.6,1.0] 

(9.110) 

Hd = 

[ 1 , 1 , 0 , 0 ] 


W = 

[1/0.05,1/0.01] 


B = 

reme z (M, F, Fid, W) 



The output vector B will contain the desired filter coefficients, i.e. 

B = [/r[0],.. .,h[M]\ (9.111) 

The corresponding impulse response and magnitude response (M =11) are shown in Figure 9.25. 
• The filter so designed has tolerances 8 ( and 8 ( that are different from 81 and 82 . However: 


§1 = 5i 

S' 8 2 


(9.112) 
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• By trial and error,we find that the required value of M necessary to meet the spec is M = 15. The 
corresponding impulse response and magnitude response are shown in Figure 9.26. 


◄ 
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Fig. 9.25 Illustration of trial design in example 9.9 (M =11). 
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Fig. 9.26 Illustration of the final design in example 9.9 (M = 15) 


© B. Champagne & F. Labeau 


Compiled November 5, 2004 





























































200 


Chapter 10 

Quantization effects 


Introduction 


Practical DSP systems use finite-precision (FP) number representations and arithmetic. The implementation 
in FP of a given LTI system with rational system function 


H(z) 


ItLo b k z~ k 

l~Lk=l a kZ~ k 


leads to deviations in its theoretically predicted performance: 

• Due to quantization of system coefficients cik s and bk s: 


//(to) =>H{(d) / //(to) 


( 10 . 1 ) 


• Due to round-off errors in finite precision arithmetic, there is quantization noise at the system output. 

• Other round-off effects such as limit cycles, etc. 

Different structures or realizations (e.g. direct form, cascade, etc.) will be affected differently by these 
effects. 


10.1 Binary number representation and arithmetic 

In a binary number representation system, a real number x £ M is represented by a sequence of binary digits 
(i.e. bits) bi £ {0,1}: 

x •<=>• .. .& 2 ^ 1 ^ 0 ^-l • ■ • 

In practice, the number of bits available for the representation of numbers is finite (e.g. 16, 32, etc.). This 
means that only certain numbers rgR can be represented exactly. 

Several approaches are available for the representation of real numbers with finite sequences of bits, some 
of which are discussed below. The choice of a specific type of representation in a given application involves 
the consideration of several factors, including: desired accuracy, system costs, etc. 
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10.1.1 Binary representations 

Signed fixed-point representations: In this type of representation, a real number x in the range [—2 K , 2 K ] 
is represented as 

x = b K b K —i ■ ■ ■ bo.b—i ■ ■ ■ b— L , b t G {0.1} (10.2) 

where bic is called sign bit or sometimes most significant bit (MSB), b j is called the least significant bit 
(LSB), the K bits b^~ i - bo represent the integer part of * and the L bits b \ ■■ ■ b / represent the fractional 
part of x. The binary dot is used to separate the fractional part from the integer part. It is convenient to 
define B = K + L so that the total number of bits is B + 1, where the 1 reflects the presence of a sign bit. 

Below, we discuss two special types of signed fixed-point representations, that is: the sign and magnitude 
(SM) and the 2’s complement (2C) representations. 


SM representation: 

format is: 


The simplest type of fixed-point representation is the SM representation. Its general 


x= (b K ---b 0 .b -1 • 


K -1 


*-l)|sm = (-1)*'x £ bp 
i=—L 


(10.3) 


2’s complement representation: The most common type of signed fixed-point representation is 2’s com¬ 
plement representation (2C). The general format is 


k- t 

x=(b K -b 0 .b -1 ■ ■ • b- L ) 2C = ~b K 2 K + £ bp 

i=-L 


(10.4) 


Example 10.1: 

► Using (10.3) and (10.4), we find that 


(0.11)sm = (0.11) 2C = 1 X 2- 1 + 1 x 2“ 2 = 3/4 

while 

(1-11) S m = (—1) x 3/4 = —3/4 
(1-11) 2C = (—1) + 3/4 = —1/4 


◄ 


Observation: For fixed-point representation, the distance between two consecutive numbers, also called 
resolution of step size, is a constant equal to 

A = 2~ l 

Thus, the available dynamic range [—2^,2^] is quantized into 2 ,<+1 uniformly spaced representation levels, 
where B = K + L. In scientific applications characterized by a large dynamic range, the use of such an 
uniform number representation grid is not the most efficient way of using the available B + 1 bits. 
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Floating-point representations: In this type of representations, a number x G M is represented as 

x = b 0 bi...b, bi+ 1 ... b t+k = M x 2 E (10.5) 

s -v--V-' 

M E 

where M is a signed mantissa and E is an exponent. Several formats do exist for binary encoding of M and 
E. 

Example 10.2: 

► THe IEEE 753 floating-point format uses 32 bvits in total. 1 is reserved for the sign, 23 for the mantissa, 
and 8 for the exponent (see Figure 10.1). The corresponding binary-to-decimal mapping is 

x = (—l) s x (0.1M) x 2 e ~ 126 

< 


K b, 


b Q b, 


8 9 


31 


-Y- 

1 bit 8 bits 


Y 

23 bits 


Fig. 10.1 Floating-point representation as per format IEEE753. 


Observations: Floating-point representations offer several advantages over fixed-point: 

• Larger dynamic range 

• Variable resolution (depends on value of E) 

However, floating-point hardware is usually more expensive, so that fixed-point is often used when system 
cost is a primary factor. 

In the sequel, we focus on fixed-point representations with small number of bits (say 16 or less), where quan¬ 
tization effects are the most significant. For simplicity, the emphasis in give to fractional representations, 
for which 

K = 0 and L = B. 

However, generalizations to mixed (i.e. integer/fractional) representations are immediate. 


10.1.2 Quantization errors in fixed-point arithmetic 

Sources of errors: The two basic arithmetic operations in any DSP system are multiplication and addition, 
as exemplified by 

y[n] = 0.5(jc[n] +x[n — 1]). 

Because only a finite number of bits is available to represent the result of these operations, internal errors 
will occur. Multiplication and addition in fixed-point lead to very different types of errors. 
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Multiplication: Consider two numbers, say a and b, each expressed in fixed-point fractional representa¬ 
tion with (B + l)-bit words: 


Cl — 0C().0C_i • • • 0C_/j 

b = Po-P-l • • • P-B 

Multiplication of these two ( B + l)-bit words in general leads to a (B' + l)-bit word, where here B' = 2 B. 
This is illustrated below: 

0.101 x 0.001 = 0.000101 

In practice, only B+ 1 bits are available to store the result, generally leading to a so-called round-off error. 
There are two basic ways in which number c = ax b with B' + 1 bits can be represented with B + 1 bits 
where B <B'\ 

• Truncation: The truncation of x with B' + 1 bits into x with B + I bits, where B < B', is expressed as 
follows: 

x = [bo-b—i... b—B') —► x= Q a [x] = ( b 0 .b-\ ...b_ B ). (10.6) 

In other words, the unwanted bits are simply dropped. 

• Rounding: The operation of rounding may be expressed as follows: 

x= (bo-b-i.-.b-B') ->x = Qm[x] = (b' 0 .b'_ l .. .b'_ B ). (10.7) 

so that \x — x\ is minimized. That is, the available number x that is closest to v is used for its represen¬ 
tation. The bits b\ (i = 0. B) may be different from the original bits 


Example 10.3: 

► Consider the representation of 5-bit number x = (0.0011) = 3/16, using only a 3-bit word (i.e. B' = 4, 
B = 2). For truncation, we have x —> x = (0.00) = 0 For rounding, we have x —> x = (0.01) = 1/4 ◄ 


Quantization error: Both rounding and truncation lead to a so-called quantization error, defined as 

e = Q[x\—x (10.8) 

The value of e depends on the type of representation being used (e.g. Sign-Magnitude, 2C), as well as 
whether rounding or truncation is being applied. For the 2C representation (most often used), it can be 
generally verified that for truncation: 


while for rounding: 


—A < e < 0 where A = 2 B 


—A/2 < e < A/2 
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Example 10.4: 

► Consider the truncation of (1.011 l) 2 c to 2+1 bits (i.e. B = 2): 

x= (1.0111) 2C = -9/16->x = gtrM = (1 -01 ) 2C = -3/4 

e=x-x= -3/4 + 9/16= -3/16 
Now consider the case of rounding: 

x= (1.0111) 2C = —9/16— >x = Q m [x] = (1.10) 2C = -1/2 
e = x — x = —1/2 + 9/16=1/16 

◄ 

Addition: Consider again two numbers, say a and b, each expressed in fixed-point fractional representa¬ 
tion with B + 1 bits. In theory, the sum of these two numbers may require up to B + 2 bits for its correct 
representation. That is: 

c = a + b = (ciCo.c_i .. .C-b) 

where c\ is a sign bit and the binary point has been shifted by one bit to the right. In practice, only 5+1 
bits are available to store the result c, leading to a type of error called overflow. 


Example 10.5: 

► Assuming 2C representation, we have: 

(1.01)+ (1.10) = -3/4-1/2= -5/4 

which cannot be represented exactly in a (2+ 1)-bit fractional 2C format. In a practical DSP system, 
the above operation would be realized using modulo-2 addition, leading to an erroneous result of 3/4 
(i.e. left cany bit being lost). ◄ 

In conventional applications of digital filters, overflow should by all mean be avoided, as it introduces 
significant distortions in the system output. In practice, two basic means are available to avoid overflow: 

• Scaling of signals at various point in a DSP system (corresponds to a left shift of the binary point; 
resolution is lost) 

• Use of temporary guard bits to the left of the binary point. 


10.2 Effects of coefficient quantization 


In our previous study of filter structures and design techniques, the filter coefficients were implicitly assumed 
to be represented in infinite precision. In practice, the filter coefficients must be quantized to a finite precision 
representation (e.g. fixed-point, 16 bits) prior to the system’s implementation. For example, consider a DFII 
realization of an HR filter with system function 


m 


Ltohz- k 

1 - lf=i a k Z~ k 


(10.9) 
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Due to quantization of the filter coefficients a* —> au and bk —> bk, the corresponding system function is no 
longer H(z) but instead 


H(z) 


LtobkZ~ k 

1-Eti a k z~ k 
H{z) + AH{z) 


( 10 . 10 ) 


Equivalently, one may think of the poles p ,t and zeros Zk of H(z) being displaced to new locations: 


Pk = Pk + Apk ( 10 . 11 ) 

z k = Zk + Azk (10.12) 


Small changes in the filter coefficients due to quantization may lead to very significant displacements of the 
poles and zeros that dramatically affect H(z), i.e. very large A H(z). 

It should be clear that quantization effects may be different for two different filter realizations with the same 
system function H(z). Indeed, even tough the two realizations have the same transfer characteristics, the 
actual system coefficients in the two realizations may be different, leading to different errors in the presence 
of quantization. 

In this section, we investigate such quantization effects. 


10.2.1 Sensitivity analysis for direct form HR filters 


Consider an HR filter with corresponding system function 


H(z) 


5U) 

A(z) 


( 10 . 13 ) 


where B(z) and A(z) are polynomial in z 1 : 

B(z) = bo-\-b\z 1 + + m 

m 

= bolKl-zx') ( 10 . 14 ) 

k= 1 

A(z) = 1 — «iz 1 - anz~ N 

M 

k= 1 

Note that the filter coefficients bk and cp uniquely define the poles and zeros of the system, and vice versa. 
Thus, any change in the a^s and Z^s will be accompanied by corresponding changes in /;^s and Zk s. 

Below, we investigate the sensitivity of the pole-zero locations to small quantization eirors in cps and b/ c s 
via a first order perturbation analysis. Note that in a direct form realization of H(z), the filter coefficients 
are precisely the «/ c s and /?/.s. Thus the results of this analysis will be immediately applicable to HR filters 
realized in DFI, DFII and DFII-transposed. 
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PZ sensitivity: In Suppose the filter coefficients in (10.13)- (10.15) are quantized to 

a = Q[ak\ = at + Acik (10.16) 

b = Q[b k \ = b k + Ab k (10.17) 

The resulting system function is now H(z) = B(z) /A(z) where 

B(z) = ba + bxz~ 1 -\ -f b M z~ M 

M 

= Mia -**- 1 ) ao.is) 

k= 1 


A(z) 


1 — d\z 1 - d N z 


-N 


M 


na-^ ‘) 

k= 1 


(10.19) 


Using the chain rule of calculus, it can be shown that to the first order in A a k , the poles of the quantized 
system become 

Pk = Pk + Ap k (10.20) 


A pic = 


v ■'N V—/* 

Li=\P k Ad/ 


( 10 . 21 ) 


U/MPk-Pl) 

A similar formula can be derived for the variation in the zeros of the quantized system as a function of A b k . 


Remarks: For a system with a cluster of two or more closely spaced poles, we have 

Pk ~ Pi for some / / k => Ap k very large 

=> AH(z) very large (10.22) 

The above problem posed by a cluster of poles becomes worse as N increases (higher probability of having 
closely spaced poles). To avoid the pole cluster problem with HR filters: 

• do not use direct form realization for large A; 

• instead, use cascade or parallel realizations with low order sections (usually in direct form II) 

As indicated above, the quantization effects on the zeros of the system are described by similar equations. 

• However, the zeros z k s are usually not as clustered as the poles p k s in practical filter designs. 

• Also, AH(z) is typically less sensitive to A z k than to A p k . 


10.2.2 Poles of quantized 2nd order system 


Intro: Consider a 2nd order all-pole filter section with 

H(z) = —— -- 9 

1 -2aiz -1 + a2Z - *- 


(10.23) 


Observe that the corresponding poles, say p\ and po, are functions of the coefficients ai and Uz- Thus, if oq 
and a 2 are quantized to B + 1 bits, only a finite number of pole locations arc possible. Furthermore, only a 
subset of these locations will correspond to a stable and causal DT system. 
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System poles: The poles of H(z) in (10.23) are obtained by solving for the roots of z 2 — 2 <X\z + 0 C 2 = 0 or 
equivalently: 

z = 00 ± \/a where A = ccf — 0 C 2 

We need to distinguish three cases, 

• A = 0: Real double pole at 

Pi = P 2 = OCt 


• A > 0: Distinct real poles at 

• A < 0: Complex conjugate poles at 


Pi 2 = oci ± Va. 


Pi ,2 = cti ±jy/\&\ = re ±]( * 


where the parameters r and 0 satisfy 


r=V 0 C 2 rcos0 = ai 


(10.24) 


Coefficient quantization: Suppose that a B+ 1-bit, fractional sign-magnitude representation is used for 
the storage of the filter coefficients a\ and ci 2 , i.e.: 

cii G {O.Zq • • bs : bi = 0 or 1} 

Accordingly, only certain locations arc possible for the corresponding poles p\ and p 2 - This is illustrated in 
Figure 10.2in the case B = 3 where we make the following observation: 

• The top part shows the quantized (a\, fl 2 )-plane, where circles correspond to A < 0 (complex conjugate 
pole locations), bullets to A = 0 and x’s to A > 0. 

• The bottom part illustrates the corresponding pole locations in the 2 -plane in the case A < 0 and A = 0 
only (distinct real poles not shown to simplify presentation). 

• Each circle in the top figure corresponds to a pair of complex conjugate location in the bottom figure 
(see e.g. circles labelled 1 , 2 , etc.) 

• According to (10.24), the complex conjugate poles are at the intersections of circles with radius yTA 
and vertical lines with ordinates rcos0 = oci. That is, distance from the origin and projection on 
real-axis arc quantized in this scheme. 

10.3 Quantization noise in digital filters 

In a digital filter, each multiplier introduces a round-off error signal, also known as quantization noise. The 
error signal from each multiplier propagates through the system, sometimes in a recursive manner (i.e. via 
feedback loops). At the system output, the round-off errors contributed from each multiplier build up in a 
cumulative way, as represented by a total output quantization noise signal. 

In this Section, we present a systematic approach for the evaluation of the total quantization noise power at 
the output of a digitally implemented LTI system. This approach is characterized by the use of an equivalent 
random linear model for (non-linear) digital filters. Basic properties of LTI systems excited by random 
sequences are invoked in the derivation. 
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a 2 



Fig. 10.2 Illustration of coefficient quantization in a second-order system.. 
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Non-linear model of digital filter: Let Oi denote an LTI filter structure with L multiplicative branches. Let 
Ui[n\, a j and v,\n] respectively denote the input, multiplicative gain and output of the zth branch (7 = 1,... ,L), 
as shown in Figure 10.3 (left). 

«,•[»] a , L-[«] k «,-[«] 

o-►-o o-►- 



Fig. 10.3 Non-linear model of digital multiplier. 

Let denote the (B + l)-bit fixed-point implementation of ‘J4\ where it is assumed that each multiplier 
output is rounded to B+ 1 bits: 


Vi[n] = CLiUi[n\ => vi[n] = Q r (fXiUj[n\) (10.25) 

2B+1 B+l 

In the deterministic non-linear model of each multiplicative branch a,- in ‘J-( is modified as shown in 
Figure 10.3 (right). 


Equivalent linear-noise model: In terms of the quantization error <?,[«], we have 

Vi[n] = Q r (ajUi[n]) 

= a jUj[n\+ei[n] (10.26) 

In the equivalent linear-noise model of each multiplicative branch a,- with quantizer Q r (.) is replaced as 
shown in Figure 10.4. 

U;[n\ a : 
o-►- 

e\n\ o 



Fig. 10.4 Linear-noise model of digital multiplier. 

The quantization error signal e t [n] (n G Z) is modelled as a white noise sequence, unconclatcd with the 
system input x[n\ and other quantization error signals ej\n\ for j / i. For a fixed n, we assume that e,-[n] is 
uniformly distributed within the interval ±^2 B , so that 

• its mean value (or DC value) is zero: 

E{ei[n\} = 0 

• its variance (or noise power) is given by: 

9-2 B 

E{ei[n] 2 } = = o 2 e 


(10.27) 


(10.28) 
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Property 1: Consider an LTI system with system function K(z)- Suppose that a zero mean white noise 
sequence e[n\ with variance a 2 is applied to its input and let f\n] denote the corresponding output (see 
Figure 10.5). Then, the sequence f[n] has zero-mean and its variance a 2 is given by 


e[n] 

o-► 


LTI system 

- K{z) - 



Fig. 10.5 Illustration of LTI system K(z) with stochastic input e[n\. 


oo 

1 \m 2 

n——oo 



|A'(co)| 2 f/co 


where k[n] denotes the impulse response of LTI system K(z). 


(10.29) 


Property 2: Let e\\n] ,..., ei[n] be zero-mean, uncorrelated white noise sequences with variance a 2 . Sup¬ 
pose that each sequence e t [n] is applied to the input of an LTI system K,(z) and let j)\n) denote the corre¬ 
sponding output. Finally, let f[n\ = f\ [n] H- fi\ n ] (see Figure 10.6). Then, the sequence f[n\ has zero-mean 

and variance 

a 2 =a 2 fl -\ -h aj L (10.30) 

where the o 2 . are computed as in Property 1. 



Fig. 10.6 LTI systems Kj(z) with stochastic inputs e,- [n] and combined output f[n\. 


Linear superposition of noise sources: Consider an LTI filter structure 9{ with L multiplicative branches. 
Assume that is implemented in fixed-point arithmetic and let ‘Jl denote the corresponding linear-noise 
model: where the noise sources e, [n] are modelled as uncorrelated zero-mean white noise sources, as previ¬ 
ously described. 

A may be interpreted as a linear system with a main input x[n], L secondary inputs e t [n] (i = 1 ,... ,L) ar| d a 
main output y[n]. Invoking the principle of superposition for linear systems, the output may be expressed as 

y[n\ = y[n] +/i [n] 4 -F/l[«] (10.31) 


In this equation: 
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Fig. 10.7 Linear-noise model of LTI system with L multipliers. 

• y[n] is the desired response in the absence of quantization noise: 

y[n] = H(z)x[n } 

where H(z) is the system function in infinite precision. 

• fi[n] is the individual contribution of noise source e,-[n] to the system output in the absence of other 
noise sources and of main input x[n]: 

fi[n\ = Ki(z)ei[n\ 

where K{(z) is defined as the transfer function between the injection point of e,\n] and the system 
output y[n] when x[n\ = 0 and ej [n] = 0 for all j / i. 


Total output noise power: According to (10.31), the total quantization noise at the system output is given 
by 

f[n] = /i[«B- K/lH 

Invoking Properties 1 and 2, we conclude that the quantization noise f[n] has zero-mean and its variance is 
given by 

° 2 f = £ l*«M| 2 (10-32) 

i—l n——oo 
j—B L oo 

= -ry£ I \ki[n}\ 2 (10.33) 

1Z i=l n=—°° 

where (10.28) has been used. The variance obtained in this way provides a measure of the quantization 
noise power at the system output. 


Note on the computation of (10.33): To compute the quantization noise power as given by (10.33), it 
is first necessary to determine the individual transfer functions Kj(z) seen by each of the noise sources <?,[/?]. 
This information can be obtained from the flowgraph of the linear-noise model ‘M by setting x[n] = 0 and 
ej[n] = 0 for all j / i. For simple system functions Kj(z), it may be possible to determine the associated 
impulse response k,[n\ via inverse z-transform and then compute a numerical value for the sum of squared 
magnitudes in (10.33). This is the approach taken in the following example. 
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Example 10.6: 

► Consider the system function 

1 

(1 -fllZ _1 )(l -C12Z~ 2 ) 

where a\ = 1/2 and a 2 = 1 /4. Let IH denote a cascade realization of H(z ) using 1st order sections, as 
illustrated in Figure 10.8 


x[n ] 

o-► 


o-►-c 

r- * - c 

5-►-9 


, -1 


it 

' Z i 

i ,r 

o- < -c 

^ _ c 

3->•-O 


-1 


a x Mj[n] 


u 2 [n\ 


y[n] 

—o 


Fig. 10.8 Cascade realization of a 2nd order system. 

Consider the fixed-point implementation of OF using a B + 1 bits fractional two’s complement (2C) 
number representation in which multiplications are rounded. In this implementation of 9~C, each product 
a,■«,■[«] in Fig. 10.8, which requires 2B + 1 for its exact representation, is rounded to B+ 1 bits: 


QiUi[n] —> Q r (ajUj[n]) = £/,-+«] +e,[n] 

25+1 bits 5+1 bits 

where e l [«] denotes round-off error. The equivalent linear-noise model for ‘Ji, shown in Fig. 10.9, may 
be viewed as a multiple input-single output LTI system. 



9-*■-s 

r-*■- c 

3— 

-1 



lr Z i 

L 

' Z 

o c 

3-«*-C 



y[n] 

-o 


-i 


n 

e 2 [n] o 


Fig. 10.9 Equivalent linear noise model. 

The output signal y[n] in Fig. 10.9 may be expressed as 

y[n)=y[n]+f[n\ 

where y[n] = H(z)x[n\ is the desired output and f[n\ represents the cumulative effect of the quantization 
noise at the system’s output. In turns, f[n] may be expressed as 

f[n\= fi[n)+f 2 [n\ 
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where 

fi[n\= Kj(z)ei[n\ 

represents the individual contribution of noise source e ( - [n] and K;(z) is the system function between the 
injection point of e,-[n] and the system output, obtained by setting the input as well as all other noise 
sources to zero. To compute the noise power contributed by source e,[n], we need to find Kj(z) and 
compute the energy in its impulse response: 

• To obtain K\ (V), set x[n] = [n\ = 0: 


Ki(z) = H(z) 

1 

(1 -aiz _1 )(l -a2Z~ 2 ) 

2 1 


The impulse response is obtained via inverse ’-transform (hence the partial fraction expansion) 
assuming a causal system: 


h [n] 




u[n] 


from which we compute the energy as follows: 


I>MI 2 

all n 




= 1.83 


• To obtain Ki{z), set x[n] = e\[n] = 0: 


Ki(z) 


1 

1 - Cl2Z~ 2 


The impulse response is 



with corresponding energy 


11*2 HI 2 

all n 



= 1.07 


Finally, according to (10.33), the total quantization noise power at the system output is obtained as : 


°/ = °dl l*i M 


s. all n 


E 1*2 w 


all n 


-2 B 


12 


(1.83 + 1.07) =2.90- 


-2B 


12 


(10.34) 


We leave it as an exercise for the student to verify that if the computation is repeated with a \ = 1/4 and 
«2 = 1/2 (i.e. reversing the order of the 1st order sections), the result will be 


o 


2 

/ 


= 3.16 


2~ 2b 

~V2 


Why...? 


◄ 
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Signal-to-quantization noise ratio (SQNR): We define the SQNR at the output of a digital system as 
follows: 

Py 

SQNR = 101og 1() in dB (10.35) 

a f 


where 


• P y denotes the average power of the output signal y[n] in infinite precision (see below). 

• denotes the average power of the total quantization noise f[n\ at the system output. This is com¬ 
puted as explained previously. 


Output signal power: The computation of the output signal power P y depends on the model used for the 
input signal x[n]. Two important cases are considered below: 

• Random input: Suppose x[n] is a zero-mean white noise sequence with variance a 2 . Then, from 
Property 1, we have 

00 rP- rn 

p y = ol E l %]| 2 = NZ / l^(w)| 2 dco (10.36) 

• Sinusoidal input: Suppose x[n] = A sin(co„/i + (|)). Then, the average power of y[n] is given by 

P v =yj//(co 0 )| 2 (10.37) 


Example 10.7: (Ex. 10.6 cont.) 


► Suppose that in Example 10.6, the input signal is a white noise sequence with sample values x{n] uni¬ 
formly distributed within ±X max , where the maximum permissible value of \x\n] |, represented by 7f max , 
may be less than 1 to avoid potential internal overflow problems. From the above, it follows that x[n] has 
zero-mean and 


The corresponding output power in infinite precision is computed from (10.35): 


00 

Py = V 2 x E l %]| 2 

n——00 


1.83a 2 


= 1.83 


y 2 
A ma 

12 


Finally, the SQNR is obtained as 
SQNR 


lOlogjo-4 


10 log 


10 


w 

1.83X 2 ax /12 

2.902- 2B /12 


6.02B + 201og 10 X max -2.01 


(dB) 


Note the dependance on the peak signal value 2f max and the number of bits B. Each additional bit con¬ 
tributes a 6dB increase in SQNR. In practice, the choice of JN l1lax is limited by overflow considerations. 

◄ 
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Further note on the computation of (10.33): The approach used in the above example for the compu¬ 
tation of £„ |&, [«] | 2 may become cumbersome if Kj(z) is too complex. For rational HR transfer functions 
Ki(z), the computation of the infinite sum can be avoided, based on the fact that 


E i%]i 2 


i 

271 



|/£(<n)| 2 dco 


It has been shown earlier that the transfer function C(z) with frequency response C((t>) = |/f,(co)| 2 has poles 
given by d k and 1 /d k , where <4 is a (simple) pole of Kj(z). Thus a partial fraction expansion of C(z) would 
yield a result in the form: 


N 


C(z)=Ci(z) + £ T 

i — i r 


Ak 


k= l 


- d k z~ 


1 


where Ci (z) is a possible FIR component, and the Ak s are partial fraction expansion coefficients. It is easily 
shown by the definition of C(z) as Kj(z)K*(1 /z*) that the partial fraction expansion corresponding to I /d k 
is indeed — A* k . Evaluating the above PFE on the unit circle and integrating over one period gives: 


5 >Ml 2 


I /-it N ] r n 

-J_c lia)da+ Z-Jj- 


Ak 


— d k e~J a l-e~j a /d* k 


}dco. 


All the above terms are easily seen to be inverse discrete-time Fourier transforms evaluated at n = 0, so that 


El*MI 2 


N 

c\ [o]+ y Ak 


k= 1 


(10.38) 


Note that the term corresponding to A* k is zero at n = 0 since the corresponding sequence has to be anti- 
causal, starting at n = — 1. 

Finally, it often occurs that two or more, noise sources e-,[n] have the same system function Ki(z). In this case, 
according to (10.33), these noise sources may be replaced by a single noise source with a scaled variance of 
c/G 2 , where q is the number of noise sources being merged. 


Example 10.8: 


► Let us study the actual effect of round-off noise on the DFII implementation of the filter designed by 
bilinear transformation in example 9.5. The coefficients of the transfer function are shown in table 9.2. 
Figure 10.10(a) shows the equivalent linear model to take quantization noise into account. The different 
2 N+ 1 noise sources can be regrouped as in Figure 10.10(b) into two composite noise sources e\ [n] and 
e' 2 [n\ with variances No 2 and (N + l)o 2 respectively. The transfer function from e' 2 [n] to the output of 
the structure is just 1, whereas the noise source e\ [n] goes through the direct part of the transfer function 
// (z)■ The corresponding output noise variances can therefore be computed as: 


for a total noise power of 


° 2 fl = i h 2 = N ° 2 e £ h M 2 

1 '«=0 n= 0 

= ° 2 e ' 2 = (N+l )G 2 , 


->-2 B oo 

Gf=-r^-(N+l+N'£h[n) 2 ). 

n =0 


(10.39) 

(10.40) 
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In the case of the coefficients shown in table 9.2, we will use the formula (10.38) in order to compute 
the sum over h[n] 2 . The first step is to create the numerator and denominator coefficients of the transfer 
function C(z) = H(z)H* (l/z*)- We will use Matlab to do so. After grouping the coefficients of the 
numerator polynomial bk in a column vector b in Matlab and doing the same for the a^s in a vector 
a, one can easily find the numerator by numc=conv (b, f lipup (con j (b) ), and the denominator 
by denc=conv (a, flipud (con j (a) ). To compute the partial fraction expansion, the function 
residuez is used, yielding: 



Ci(z) = 

9.97 10“' 


II 

0.23463, 

so that the actual sum 

X>N 2 = 

72=0 

0.23464. 


Finally, the output noise variance is given by: 


aj = 0.70065 2- 2B . 

For instance, assuming a 12-bit representation and an input signal uniformly distributed between — ^ 
and 4. we would get an SQNR of 50.7 dB. Though this value might sound high, it will not be sufficient 
for many applications (e.g. “CD quality” audio processing requires a SNR of the order of 95 dB). 

Note that the example is not complete: 

- the effect of coefficient quantization has not been taken into account. In particular, some low 
values of the number B + 1 of bits used might push the poles outside of the unit circle and make 
the system unstable. 

- the effect of overflows has not been taken into account; in practise, overflow in key nodes is 
avoided by a proper scaling of the input signal by a factor s < 1 such that the resulting signals in the 
nodes of interest do not overflow. The nodes to consider are final results of accumulated additions, 
since overflow in intermediate nodes is harmless in two’s complement arithmetics. These nodes 
are, in the case of Figure 10.10(b), the two nodes where the noise components are injected. By 
looking at the transfer function from the input to these nodes, it is easy to find a conservative value 
of s that will prevent overflow. Note that this will also reduce the SQNR, since the power l\ will 
be scaled by a factor s 2 . 

Finally, let us point out that if we have at our disposal a double-length accumulator (as is available in 
many DSP chips, see chapter ??), then the rounding or truncation only has to take place before storage in 
memory. Consequently, the only locations in the DFII signal flow graph where quantization will occur 
are right before the delay line, and before the output, as illustrated in Figure 10.11. Carrying out a study 
similar to the one outline above, we end up with a total output noise variance of 

9-2 B 

°/= -ft ( 1 + I>M 2 )> 

72=0 

where the factors N have disappeared. With 12 bits available, this leads to an SQNR value of 59.02 dB. 

◄ 


10.4 Scaling to avoid overflow 

Problem statement: Consider a filter structure 9{ with K adders and corresponding node values w, [n], 
as illustrated in Fig. 10.12. Note that whenever |w,[n]| > 1, for a given adder at a given time, overflow 
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e N+ lM 

O 


H>] K 

.»’["] x[n] 

e d n \ 


' z 1 i 

b x 

1 ► 0 0 -»■- 

e V+2t' 7 ] 

ejv-i [«] 

a »-l 

V 1 

e 2JV [”] 

e N [n\ 

O-►- 

a N 

-<- 

. -1 
r z 

b N 

-►- 

e 2V+lt ,? ] 

--«— 0 c 



e'2 m= 

i=JV+l 

-o 


(a) 



Fig. 10.10 DFII modified to take quantization effects into account (a) each multiplier contains 
a noise source and (b) a simplified version with composite noise sources. 



a jv-i 

b N-l 

a N 

0 - -c 

•z- 1 

K 

-►- 0 


Fig. 10.11 Round-off noise model when a double-length accumulator is available. 
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x[h] 



Fig. 10.12 Linear-noise model of LTI system with L multipliers. 


will result in the corresponding fixed-point implementation. Overflow usually produces large, unpredictable 
distortion in the system output and must be avoided at all cost. 


Principles of scaling: The basic idea is very simple: properly scale the input signal x[n] to a range \x\n\ \ < 
X max so that the condition 

K-[«]|<1, (10.41) 

is enforced at all time n. Several approaches exist for choosing X max , the maximum permissible value of 
x[n]. The best approach usually depends on the type of input signals being considered. Two such approaches 
are presented below: 


Wideband signals: For wideband signals, a suitable value of X max is provided by 


X„ 


< 


max,X„ \hi[n\\ 


(10.42) 


where H;(z) denotes the transfer function from the system input to the ith adder output. The above choice 
of X max ensures that overflow will be avoided regardless of the particular type of inputs: it represents a 
sufficient condition to avoid overflow. 


Narrowband signals: For narrowband signals, the above bound on X max may be too conservative. An 
often better choice of A max if provided by the bound 


X„ 


< 


max 


I,CO 




(10.43) 
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Chapter 11 


Fast Fourier transform (FFT) 


Computation of the discrete Fourier transform (DFT) is an essential step in many signal processing algo¬ 
rithms. Even for data sequences of moderate size N, a direct computation of the /V-point DFT requires 
on the order of N 2 arithmetic operations. Even for moderate values of N, this usually entails significant 
computational costs. 

In this Chapter, we investigate so-called fast Fourier transform (FFT) algorithms for the efficient computa¬ 
tion of the DFT. These algorithms achieves significant savings in the DFT computation by exploiting certain 
symmetries of the Fourier basis functions and only require on the order of Nlog 2 N arithmetic operations. 
They arc at the origin of major advances in the field of signal processing, particularly in the 60’s and 70’s. 
Nowadays, FFT algorithms find applications in numerous commercial DSP-based products. 

11.1 Direct computation of the DFT 

Recall the definition of the /V-point DFT of a discrete-time signal x[n\ defined for 0 < n < N — 1: 

N-\ 

X[k]=Y,4n]W^ n , k = 0,l,...,N-l (11.1) 

n —0 

where for convenience, we have introduced 


W N = e~ j2n/N , (11.2) 

Based on (11.1), one can easily come up with the following algorithmic steps for its direct evaluation, also 
known as the direct approach: 


Step 1: Compute W l N and store it in a table: 

W l N = e~^ N 

= cos(2nl/N)+jsm(2nl/N), l = 0,1, 1 


(11.3) 


Note: this only needs to be evaluated for / = 0,1, 1 because of the periodicity. 
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Step 2: Compute the DFT X[k] using stored values of W l N and input data x[n \: 

for k = 0 : fV — 1 (11.4) 

X[k}*-x[ 0] 
for n = 1 : N - 1 
/ = ( kn)N 

X[k] <-X[k]+x[n]Wff 

end 

end 


The complexity of this approach can be broken up as follows: 

• In step (1): N evaluation of the trigonometric functions sin and cos (actually less than N because of 
the symmetries). These values are usually computed once and stored for later use. We refer to this 
approach as table look-up. 

• In step (2): 

- There are N(N — 1) « N 2 complex multiplications (<8> c ) 

- There arc N(N — I) ~ /V 2 complex additions (® c ) 

- One must also take into account the overhead: indexing, addressing, etc. 

Because of the figures above, we say that direct computation of the DFT is 0(N 2 ), i.e. of order N 2 . The 
same is true for the IDFT. 

In itself, the computation of the /V-point DFT of a signal is a costly operation. Indeed, even for moderate 
values of N, the 0(N 2 ) complexity will usually require a considerable amount of computational resources. 

The rest of this Chapter will be devoted to the description of efficient algorithms for the computation of the 
DFT. These so-called Fast Fourier Transform or FFT algorithms can achieve the same DFT computation in 
only 0(N\og 2 N). 

11.2 Overview of FFT algorithms 

In its most general sense, the term FFT refers to a family of computationally fast algorithms used to compute 
the DFT. FFT should not be thought of as a new transform: FFTs are merely algorithms. 

Typically, FFT algorithms require D(A r log 2 N) complex multiplications (<8> c ), while the direct evaluation of 
the DFT requires 0(N 2 ) complex multiplications. For /V I, /Vlog 2 N <C N 2 so that there is a significant 
gain with the FFT. 


Basic principle: The FFT relies on the concept of divide and conquer. It is obtained by breaking the DFT 
of size N into a cascade of smaller size DFTs. To achieve this, two essential ingredients are needed: 

• N must be a composite number (e.g. N = 6 = 2 x3). 
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• The periodicity and symmetry properties of the factor Wfj in (11.2) must be exploited, e.g.: 



(11.5) 

( 11 . 6 ) 


Different types of FFTs: There are several FFT algorithms, e.g.: 

• IV = 2 V => radix-2 FFTs. These are the most commonly used algorithms. Even then, there are many 
variations: 

- Decimation in time (DIT) 

- Decimation in frequency (DIF) 

• IV = r v =>■ radix-r FFTs. The special cases r = 3 and r = 4 are not uncommon. 

• More generally, N = p\Pi---Pi where the /;,s are prime numbers lead to so-called mixed-radix FFTs. 

Radix-2 FFTs: In these algorithms, applicable when N = 2 V : 

• the DFT/v is decomposed into a cascade of v stages; 

• each stage is made up of j 2-point DFTs (DFT 2 ). 

Radix-2 FFTs are possibly the most important ones. Only in very specialized situations will it be more 
advantageous to use other radix type FFTs. In these notes, we shall mainly focus on the radix-2 FFTs. 
However, the students should be able to extend the basic principles used in the derivation of radix-2 FFT 
algorithms to other radix type FFTs. 

11.3 Radix-2 FFT via decimation-in-time 

The basic idea behind decimation-in-time (DIT) is to partition the input sequence x[n\, of length N = 2 V , 
into two subsequences, i.e. x[2r] and x[2r + 1], r = 0,1,..., (A^/2) — 1, corresponding to even and odd 
values of time, respectively. It will be shown that the IV-point DFT of x[n\ can be computed by properly 
combining the (IV/2)-point DFTs of each subsequences. In turn, the same principle can be applied in the 
computation of the (IV/2)-point DFT of each subsequence, which can be reduced to DFTs of size N/4. This 
basic principle is repeated until only 2-point DFTs are involved. The final result is an FFT algorithm of 
complexity 0(Nlog 2 N) 

We will first review the 2-point DFT, which is the basic building block of radix-2 FFT algorithms. We 
will then explain how and why the above idea of decimation-in-time can be implemented mathematically. 
Finally, we discuss particularities of the resulting radix-2 DIT FFT algorithms. 

The 2-point DFT: In the case N = 2, (11.1) specializes to 


X[k}=x[0}+x[l}Wl k = 0,1. 


(11.7) 
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Since Wi = e ;7t = — 1, this can be further simplified to 


X[0] = jc[0]+jc[1] 

m = ^[o] —•«[!], 


( 11 . 8 ) 

(11.9) 


which leads to a very simple realization of the 2-point DFT, as illustrated by the signal flowgraph in Fig¬ 
ure 11.1. 



Fig. 11.1 Signal flow-graph of a 2-point DFT. 


Main steps of DIT: 

(1) Split the in (11.1) into eV en odd 

(2) Express the sums even and £„ odd as (/V/2)-point DFTs. 

(3) If N/2 = 2 stop; else, repeat the above steps for each of the individual (/V/2)-point DFTs. 


Case IV = 4 = 2 2 : 


• Step (1): 

X[k] = x[0]+x[l]Wt+x[2]W? + x[3]Wi k 
= (jc[0] +x[2]w2 k )+Wfc[l] +x[3]Wi k ) 

• Step (2): Using the property W^ k = W k , we can write 

X[k] = (x[0]+x[2]W$)+Wfc[l]+x[3]wZ) 
= G[k] + WfH[k] 

G[k\ = DFT 2 {even samples} 

H[k\ = DFT 2 {odd samples} 

Note that G[k] and H[k\ are 2-periodic, i.e. 

G[k + 2] = G[k\, H[k + 2}=H[k\ 


( 11 . 10 ) 


( 11 . 11 ) 

( 11 . 12 ) 

(11.13) 


(11.14) 


• Step (3): Since N/2 = 2, we simply stop; that is, the 2-point DFTs G[k\ and H\k\ cannot be further 
simplified via DIT. 
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The 4-point DFT can thus be computed by properly combining the 2-point DFTs of the even and odd 
samples, i.e. G[k] and H[k\, respectively: 

X[k] = G[k]+WfH[k], k = 0,1,2,3 (11.15) 

Since G[k] and H\k\ arc 2-periodic, they only need to be computed for k = 0,1, hence the equations: 

Xo [k] = G[0] + lT 4 0 //[0] 

X { [k] = G[l} + WjH[l] 

X 2 [k} = G[2} + WiH[2] =G[0} + wZh[0] 

X 3 [k] = G[3] + WfH[3\=G[l} + WjH[l} 

The flow graph corresponding to this realization is shown in Figure 11.2. Note the special ordering of the 
input data. Here further simplifications of the factors are possible here: W® = I, W 4 = —j, W 4 2 = — 1 
and W% = j. 



X[0] 

X[l] 

X[2] 

X[3] 


Fig. 11.2 Decimation in time implementation of a 4-point DFT. The DFT 2 blocks are as 
shown in figure 11.1. 


General case: 

• Step (1): Note that the even and odd samples of x[n\ can be represented respectively by the sequences 
x[2r] and x[2r+ 1], where the index r now runs from 0 to y — 1. Therefore 

m = I>[n] W N 

n —0 

N __ j - — l 

= £ x[2r}wi r + Wk £ x[2 r+ 1 }w£ kr (11.16) 

r =0 r =0 


• Step (2): Using the property W^ kr = we obtain 

K — \ — — \ 

X[k} = Y^AK,2 + W k N Yx[2r+X]W^ 

r =0 r=0 

= G[k]+W$H[k] 


(11.17) 
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where we define 

G[k] 4 DFT iV/2 {x[2r],r = 0,l...,^-l} (11.18) 

H[k] 4 DFT iV/2 {x[2r+l],r = 0,...,^-l} (11.19) 

Note that G[k] and H\k\ are ^-periodic and need only be computed for k = 0 up to ^ — 1. Thus, for 
k>N/2, (11.17) is equivalent to 

X[k] = G{k- N 2 \ + W k N H{k- N 2 } (11.20) 

• Step (3): If N/2 > 4, apply these steps to each DFT of size N/2. 


Example 11.1: Case IV = 8 = 2 3 

► The application of the general decimation in time principle to the case N = 8 is described by Figures 11.5 
through 11.5, where, for the sake of uniformity, all phase factors have been expressed as W% (e.g. Vty 1 = 

wi). 

Figure 11.3 illustrates the result of a first pass through DIT steps (1) and (2). The outputs of the top 
(bottom) 4-point DFT box are the coefficients G[k] (resp. H\k\) for k = 0,1,2,3. Note the ordering of 
the input data to accommodate the 4-point DFTs over even and odd samples. In DIT step (3), since 



Fig. 11.3 Decomposition of an 8-point DFT in 2 4-point DFTs. 

N /2 = 4, the DIT procedure is repeated for the computation of the 4-point DFTs G\k] nd H[k\. This 
amounts to replacing the 4-point DFT boxes in Figure 11.3 by the flowgraph of the 4-point DIT FFT, as 
derived previously. In doing so, the order of the input sequence must be modified accordingly. The result 
is illustrated in Figure 11.4. The final step amount to replacing each of the 2-point DFTs in Figure 11.4 
by their corresponding signal flowgraph. The result is illustrated in Figure 11.5. ◄ 
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Computational complexity: The DIT FFT flowchart obtained for N = 8 is easily generalized to /V = 2 V , 
i.e. an arbitrary power of 2. In the general case, the signal flow graph consists of a cascade of v = log 2 N 
stages. Each stage is made up of N/2 basic binary flow graphs called butterflies, as shown in Figure 11.6. 
In practice, a simplified butterfly structure is used instead of the one in Figure 11.6. Realizing that 

w r N +N/ 2 = w^ /2 = = -W r N , 

an equivalent butterfly with only one complex multiplication can be implemented, as shown in Figure 11.7. 
By using the modified butterfly structure, the computational complexity of the DIT FFT algorithm can be 
determined based on the following: 

• there are v = log 2 ./V stages; 

• there are N/2 butterflies per stage; 

• there is 1 <g> c and 2 © c per butterfly. 

Hence, the total complexity: 

N 

— \og 2 N® c , N\og 2 N® c (11.21) 


Ordering of input data: The input data x[n] on the LHS of the FFT flow graph is not listed in standard 
sequential order (refer to Figures 11.2 or 11.5). Rather, the data is arranged is a so-called bit-reversed order. 
Indeed, if one looks at the example in Figure 11.5 and compares the indices of the samples x[n] on the left 
of the structure with a natural ordering, one sees that the 3-bit binary representation of the actual index is 
the bit-reversed version of the 3-bit binary representation of the natural ordering: 
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Fig. 11.6 A butterfly, the basic building block of an FFT algorithm. 


Sample index 

Actual offset in 

memory 

decim. 

binary 

decim. 

binary 

0 

000 

0 

000 

4 

100 

1 

001 

2 

010 

2 

010 

6 

110 

3 

Oil 

1 

001 

4 

100 

5 

101 

5 

101 

3 

Oil 

6 

110 

7 

111 

7 

111 


Practical FFT routines contain bit-reversing instructions, either prior or after the FFT computation, depend¬ 
ing upon the specific nature of the algorithm. Some programmable DSP chips contain also an option for 
bit-reversed addressing of data in memory. 

We note that the DIT FFT flowgraph can be modified so that the inputs are in sequential order (e.g. by 
properly re-ordering the vertical line segments in Fig. 11.3); but then the output X[k] will be listed in 
bit-reversed order. 
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Fig. 11.7 A modified butterfly, with only one multiplication. 


Storage requirement: A complex array of length A, say A \q\ (q = 0 1), is needed to store the input 

data x[n\. The computation can be done in-place, meaning that the same array A [q\ is used to store the results 
of intermediate computations. This is made possible by the very structure of the FFT flow graph, which is 
entirely made up of independent binary butterfly computations. We note that once the outputs of a particular 
butterfly have been computed, the corresponding inputs are no longer needed and can be overwritten. Thus, 
provided a single additional complex register is made available, it is possible to overwrite the inputs of each 
butterfly computation with the corresponding outputs. Referring to Figure 11.7 and denoting by A\ and A 2 
the storage locations of the two inputs to the butterfly, we can write the pseudo-code of the butterfly: 

tmp <— A 2 * W]y 

A 2 <— Ai — tmp 
Ai <— Ai+tmp, 

after which the input has been overwritten by the output, and use of the temporary storage location tmp has 
been made. This is so because the inputs of each butterfly are only used once. 


FFT program: Based on the above consideration, it should be clear that a properly written program for 
the DIT FFT algorithm contains the following ingredients: 

• Pre-processing to address the data in bit-reversed order; 

• An outer loop over the stage number, say from / = 1 to v = log 2 /V. 

• An inner loop over the butterfly number, say from k = 1 to A/2. Note that the specific indices of the 
butterfly inputs depends on the stage number /. 

11.4 Decimation-in-frequency 

Decimation in frequency is another way of decomposing the DFT computation so that the resulting algorithm 
has complexity 0(Alog 2 A). The principles are very similar to the decimation in time algorithm. 

Basic steps: Assume A = 2 V , where v integer > 1. 

(1) Partition DFT samples X[k] {k = 0,1,... ,A— 1) into two subsequences: 

- even-indexed samples: X[2r\,r = 0,1,..., j — 1 
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- odd-indexed samples: X[2r+ 1],r = 0,1,^ — 1 

(2) Express each subsequence as a DFT of size N/2. 

(3) If N/2 = 2, stop; else, apply steps (l)-(3) to each DFT of size N/2. 


CaseiV = 4: 

• For even-indexed DFT samples, we have 

3 

X[2 r\ = £ x[n}Wl r '\ r = 0,1 

n— 0 

= jc[0 ]W 4 ° +x[\]wZ r + x[2]wf r + x[3]W^ r 

Now, observe that: 

W 4 ° = 1 W} r = W[ 

wt = 1 wt = wl r w} r = W{ 

Therefore 

X[2r\ = (x[0]+x[2]) + (x[l]+x[3])W 2 r 
= u[0}+u[l}WZ 
= DFT 2 {w[0],w[1]} 

u[r]=x[r]+x[r + 2], r = 0,1 

• In the same way, it can be verified that for the odd-indexed DFT samples, 

X[2r+l] = DFT 2 {v[0],v[l]} 
v[r] = WJ (x[r] — x[r + 2]), r = 0,1 

• The resulting signal flow graph is shown in Figure 11.8, where each of the 2-point DFT have been 
realized using the basic SFG in Figure 11.7. 

General case: Following the same steps as in the case N = 4, the following mathematical expressions can 
be derived for the general case N = 2 V : 

X[2r\ = DFT N/2 {u[r]} (11.22) 

X[2r +1] = DFTy /2 {v[r]} (11.23) 

where r = 0,1,... ,N/2 — 1 and 

u[r]=x[r]+x[r + N/2] (11.24) 

v[r] = W^{x[r) —x[r + 2]) (11.25) 
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Fig. 11.8 Decimation in frequency realization of a 4-point DFT. 


Remarks on the DIF FFT: Much like the DIT, the DIF radix-2 FFT algorithms have the following char¬ 
acteristics: 

• computational complexity of 0(Nlog 2 N)', 

• output X[k] in bit-reversed order; 

• in-place computation is also possible. 

A program for FFT DIF contains components similar to those described previously for the FFT DIT. 


11.5 Final remarks 

Simplifications: The FFT algorithms discussed above can be further simplified in the following special 
cases: 

- x[n] is a real signal 

- only need to compute selected values of X \k] (=^ FFT pruning) 

- x[n] containing large number of zero samples (e.g. as a result of zero-padding) 

Generalizations: The DIT and DIF approaches can be generalized to: 

- arbitrary radix (e.g. N = 3 V and N = 4 V ) 

- mixed radix (e.g. N = pipo - ■ ■ Pk where p, are prime numbers) 
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Chapter 12 

An introduction to Digital Signal Processors 


12.1 Introduction 

Digital Signal Processing Chips are specialized microprocessors aimed at performing DSP tasks. They are 
often called Digital Signal Processors — “DSPs” for hardware engineers. These DSP chips are sometimes 
referred to as PDSPs (Programmable Digital Signal Processor), as opposed to hard-wired solutions (ASICs) 
and reconfigurable solutions (FPGAs). 

The first DSP chips appeared in the early 80’s, mainly produced by Texas Instruments (TI). Nowadays, their 
power has dramatically increased, and the four big players in this game are now TI, Agere (formerly Lucent), 
Analog Devices and Motorola. All these companies provide families of DSP processors with different target 
applications - from low-cost to high performance. 

As the processing capabilities of DSPs have increased, they are being used in more and more applications. 
Table 12.1 lists a series of application areas, together with the DSP operations they require, as well as a 
series of examples of devices or appliances where a DSP chip can be used. 

The very need for specialized microprocessors to accomplish DSP-related tasks comes from a series of 
requirements of classic DSP algorithms, that could not be met by general purpose processors in the early 
80's: DSPs have to support high-performance, repetitive and numerically intensive tasks. 

In this chapter, we will review the reasons why a DSP chip should be different from an other microprocessor, 
through motivating examples of DSP algorithms. Based on this, we will analyze how DSP manufacturers 
have designed their chips so as to meet the DSP constraints, and review a series of features common to most 
DSP chips. Finally, we will briefly discuss the performance measures that can be used to compare different 
DSP chips. 


12.2 Why PDSPs ? 


To motivate the need for specialized DSP processors, we will study in this section the series of operations 
required to accomplish two basic DSP operations, namely FIR filtering and FFT computation. 
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Application Area 

DSP task 

Device 

Speech & Audio Signal Pro¬ 
cessing 

Effects (Reverb, Tone Control, 
Echo) , Filtering, Audio Compres¬ 
sion, Speech Synthesis, Recogni¬ 
tion & Compression, Frequency 
Equalization, Pitch , Surround 
Sound 

Musical instruments & Amplifiers, 
Audio Mixing Consoles & Record¬ 
ing Equipment, Audio Equipment 
& Boards for PCs, Toys & Games, 
Automotive Sound Systems, DAT 
& CD Players, HDTV Equipment, 
Digital Tapeless Recorders, Cellu¬ 
lar phones 

Instrumentation and Mea¬ 
surement 

Fast Fourier Transform (FFT), Fil¬ 
tering, Waveform Synthesis, Adap¬ 
tive Filtering, High Speed Numeric 
Calculations 

Test & Measurement Equipment, 
I/O Cards for PCs, Power Meters, 
Signal Analyzers and Generators 

Communications 

Modulation & Transmission, De¬ 
modulation & Reception, Speech 
Compression, Data Encryption, 
Echo Cancellation 

Modems, Fax Machines, Broadcast 
Equipment, Mobile Phones, Digi¬ 
tal Pagers, Global Positioning Sys¬ 
tems, Digital Answering Machines 

Medical Electronics 

Filtering, Echo Cancellation, Fast 
Fourier Transform (FFT), Beam 
Forming 

Respiration, Heart & Fetal Mon¬ 
itoring Equipment, Ultra Sound 
Equipment, Medical Imaging 
Equipment, Hearing Aides 

Optical and Image Process¬ 
ing 

2-Dimensional Filtering, Fast 
Fourier Transform (FFT), Pattern 
Recognition, Image Smoothing 

Bar Code Scanners, Automatic 
Inspection Systems, Fingerprint 
Recognition, Digital Televisions, 
Sonar/Radar Systems, Robotic Vi¬ 
sion 


Table 12.1 A non-exhaustive list of applications of DSP chips. 
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FIR Filtering: Filtering a signal x[n] through an FIR filter with impulse response h[n\ amounts to com¬ 
puting the convolution 

y[n] = h[n ] **[«]. 

At each sample time n, the DSP processor has to compute 


N-\ 


y[n\ = Y h[k]x[n-k], 

k =0 


where N is the length of the FIR. From an algorithmic point of view, this means that the processor has to fetch 
N data samples at each sample time, multiply them with corresponding filter coefficients stored in memory, 
and accumulate the sum of products to yield the final answer. Figure 12.1 illustrates these operations at two 
consecutive times n and n + 1. 

It is clear from this drawing that the basic DSP operation in this case is a multiplication followed by an 
addition, also know as a Multiply and Accumulate operation, or MAC. Such a MAC has to be implemented 
N times for each output sample. 

This also shows that the FIR filtering operation is both data intensive (N data needed per output sample) 
and repetitive (the same MAC operation repeated N times with different operands). 

An other point that appears from this graphic is that, although the filter coefficients remain the same from 
one time instant to the next, the input coefficients change. They do not all change: the last one is discarded, 
the newest input sample appears, and all other samples are shifted one location in memory. This is known 
as a First In-First Out (FIFO) structure or queue. 

A final feature that is common to a lot of DSP applications is the need for real time operation: the output 
samples y[n] must be computed by the processor between the moment x[n\ becomes available and the time 
x[n + 1] enters the FIFO, which leaves T s seconds to carry out the whole processing. 


FFT Computation: As explained in section 11.3, a classical FFT computation by a decimation-in-time 
algorithm uses a simple building block known as a butterfly (see Figure 11.7). This simple operation on 2 
values involves again multiplication and addition, as suggested by the pseudo-code in section 11.3. An 
other point mentioned in section 11.3 is that the FFT algorithm yields DFT coefficient in bit-reversed order, 
so that once they are computed, the processor has to restore the original order. 


Summary of requirements: We have seen with the two above examples that DSP algorithms involve: 

1. Real time requirements with intensive data flow through the processor; 

2. Efficient implementation of loops, and MAC operations; 

3. Efficient addressing schemes to access FIFOs and/or bit-reversed ordered data. 

In the next section we will describe the features common to many DSPs that address the above list of 
requirements. 
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Fig. 12.1 Operations required to compute the output of a 7-tap FIR filter at times n and n + 1. 


12.3 Characterization of PDSPs 
12.3.1 Fixed-Point vs. Floating-Point 

Programmable DSPs can be characterized by the type of arithmetic they implement. Fixed-point processors 
are in general much cheaper than their floating-point counterparts. However, the dynamic range offered 
by the floating-point processors is much higher. An other advantage of floating-point processors is that 
they do not require the programmer to worry too much about overflow and scaling issues, which are of 
paramount importance in fixed-point arithmetics (see section 10.1.2). Some applications require a very low 
cost processor with low power dissipation (like cell phones for instance), so that fixed-point processors have 
the advantage on these markets. 


12.3.2 Some Examples 

We present here a few examples of existing DSPs, to which we will refer later to illustrate the different 
concepts. 

Texas Instrument has been successful with its TMS320 series for nearly 20 years. The latest generation of 
these processors is the TMS320C6xxx family, which represent very high performance DSPs. This family 
can be grossly divided in the TMS320C62xx, which are fixed-point processors, and the TMS320C67xx, 
which use floating-point arithmetics. 

An other example of a slightly older DSP is the DSP56300 from Motorola. The newer, state-of-the-art, DSPs 
from this company are based on the very high performance StarCore DSP Core, which yield unprecedented 
processing power. This is the result of a common initiative with Agere systems. 

Analog Devices produces one of the most versatile low-cost DSPs, namely the ADSP21xx family. The next 
generation of Analog Devices DSPs are called SHARC DSPs, and contain the ADSP21xxx family of 32-bit 
floating-point processors. Finally, their latest generation is called TigerSHARC, and provides both fixed 
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and floating point operation, on variable word lengths. The TigerSHARC is a very high performance DSP 
targeting mainly very demanding applications, such as telecommunication networks equipment. 


12.3.3 Structural features of DSPs 

We will now review several features of Digital Signal Processors that are designed to meet the demanding 
constraints of real-time DSP. 


Memory Access: Harvard architecture 

Most DSPs are created according to the so-called Harvard Architecture. In classic microprocessors, there is 
only one memory bank, one address bus and one data bus. Both program instructions and data are fetched 
from the same memory through the same buses, as illustrated in Figure 12.2(a). On the other hand, in 
order to increase data throughput, DSP chips have in general two separate memory blocks, one for program 
instructions and one for data. This is the essence of the Harvard architecture, where each memory block 
also has a separate set of buses, as illustrated in Figure 12.2(b). Combined with pipelining (see below), this 
architecture enables simultaneous fetching of the next program instruction while fetching the data for the 
current instruction, thus increasing data throughput. This architecture enables two memory accesses during 
one clock cycle. 

Many DSP chips have an architecture similar to the one in Figure 12.2(b), or an enhanced version of it. 
A common enhancement is the duplication of the data memory: many common DSP operations use two 
operands, like e.g. the MAC operation. It is thus normal to imagine fetching two data words from memory 
while fetching one instruction. This is realized in practise either with two separate data memory banks with 
their own buses (like in the Motorola DSP563xx family— see Figure 12.3), or by using two-way accessible 
RAM (like in Analog Device’s SHARC Family), in which case only the buses have to be duplicated. Finally, 
other enhancements found in some DSPs (such as SHARC processors) is a program instruction cache. 

Other critical components that enable an efficient flow of data on and off the DSP chip are Direct Memory 
Access (DMA) units, which enable direct storage of data in memory without processor intervention. (See 
e.g. the architecture of the Motorola DSP56300 in Figure 12.3). 


Specialized hardware units and instruction set 

In order to accommodate the constraints of DSP algorithms, DSP chips usually comprise specialized hard¬ 
ware units not found in many other processors. 

Most if not all DSP chips contain one or several dedicated hardware multipliers, or even MAC units. On 
conventional processors without hardware multipliers, binary multiplication can take several clock cycles; 
in a DSP, it only takes one clock cycle. The multiplier is associated with an accumulator register (in a 
MAC unit), which normally is wider than a normal register. For instance, the Motorola 563xx family 
(see Figure 12.3) contains a 24-bit MAC unit, and a 56-bit accumulator. The extra bits provided by the 
accumulator are guard bits that prevent overflow from occurring during repetitive accumulate operations, 
such as the computation of the output of an FIR filter. In order to store the accumulated value into a normal 
register or in memory, the accumulator value must be shifted right by the number of guard bits used, so that 
resolution is lost, but no overflow occurs. Figure 12.4 illustrates the operation of a MAC unit, as well as the 
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(a) (b) 


Fig. 12.2 Illustration of the difference between a typical microprocessor memory access (a) 
and a typical Harvard architecture (b). 


need for guard bits. Note that G guard bits prevent overflow in the worst case of the addition of 2° words. 
The issue of shifting the result stored in the accumulator is very important: the programmer has to keep 
track of the implied binary point, depending on the type of binary representation used (integer or fractional 
or mixed). The instruction set of DSP chips often contains a MAC instruction that not only does the MAC 
operation, but also increments pointers in memory. 

We have also seen that DSP algorithms often contain repetitive instructions. Remember that to compute one 
output sample of an /V-tap FIR filter, you need to repeat N MAC operations. This is why DSPs contain a 
provision for efficient looping (called zero-overhead looping), which may be a special assembly language 
instruction — so that the programmer does not have to care about checking and decrementing loop counters 
— , or even a dedicated hardware unit. 

The third component that is necessary to DSP algorithms implementation is an efficient addressing mecha¬ 
nism. This is often taken care of by a specialized hardware Address Generation Unit. This unit works in the 
background, generating addresses in parallel with the main processor computations. Some common features 
of a DSP’s address generation unit are: 

- register-indirect addressing with post-increment, where a register points to an address in memory, 
from which the operand of the current instruction is fetched, and the pointer is incremented automat¬ 
ically at the end of the instruction. All of this happens in one instruction cycle. This is a low-level 
equivalent of a C language * ( var_name++) instruction. It is obvious that such a function is very 
useful when many contiguous locations in memory have to be used, which is common when process¬ 
ing signal samples. 

- circular addressing, where addresses are incremented modulo a given buffer size. This is an easy way 
to implement a FIFO or queue, which is the best way of creating a window sliding over a series of 
samples, as in the FIR filtering example given above. The only overhead for the programmer is to set 
up the circular addressing mode and the buffer size in some control register. 
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Fig. 12.3 Simplified architecture of the Motorola DSP563xx family. ( Source: Motorola[9]). 
The DSP563xx is a family of 24-bit fixed-point processors. Notice the 3 separate memory 
banks with their associated sets of buses. Also shown is the DMA unit with its dedicated bus 
linked directly to memory banks and off-chip memory. 
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Fig. 12.4 Operation of a MAC unit with guard bits. Two A-bit operands are multiplied, 
leading to a 2A-bit result, which is accumulated in a (2A+ G)-bit accumulator register. The 
need for guard bits is illustrated with the given operands: the values of the accumulator at 
different times are shown on the right of the figure, showing that the fourth operation would 
generate an overflow without the guard bits: the positive result would appear as a negative 
value. It is the programmer’s task in this case to keep track of the implied binary point when 
shifting back the accumulator value to an /V-bit value. 


- bit-reversed addressing, where address offsets are bit-reversed. This has been shown to be useful for 
the implementation of FFT algorithms. 


Parallelism and Pipelining 

At one moment, it becomes difficult to increase clock speed enough to satisfy the ever more demanding DSP 
algorithms. So, in order to meet real-time constraints in more and more sophisticated DSP applications, 
other means of increasing the number of operations carried out by the processor have been implemented: 
parallelism and pipelining. 


Pipelining: Pipelining is a principle that is in application in any modern processor, and is not limited to 
DSPs. Its goal is to make better use of the processor resources by avoiding leaving some components idle 
while others are working. Basically, the execution of an instruction requires three main steps, each carried 
out by a different part of the processor: a fetching step, during which the instruction is fetched from memory; 
a decoding step, during which the instruction is decoded and an execution step during which it is actually 
executed. Often, each of these three steps can also be decomposed in sub-step: for example, the execution 
may require fetching operands, computing a result and storing the result. The idea behind pipelining is 
just to begin fetching the next instruction as soon as the current one enters the decode stage. When the 
current instruction reaches the execution stage, the next one is a the decode stage, and the instruction after 
that is in the fetching stage. In this way, the processor resources are used more efficiently and the actual 
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instruction throughput is increased. Figure 12.5 illustrates the pipelining principle. In practise, care must 
be taken because instruction do not all require the same time (mostly during the execution phase), so that 
NOPs should sometimes be inserted. Also, branching instructions might ruin the gain of pipelining for a 
few cycles, since some instructions already in the pipeline might have to be dropped. These issues are far 
beyond the scope of this text and the interested reader is referred to the references for details. 
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Fig. 12.5 Schematic illustration of the difference in throughput between a pipelined (right) 
and non pipelined (left) architecture. Columns represent processor functional units, lines rep¬ 
resent different time instants. Idle units are shaded. 


Data level parallelism: Data level parallelism is achieved in many DSPs by duplication of hardware 
units. It is not uncommon in this case to have two or more ALUs (Arithmetic and Logic Unit), two or more 
MACs and Accumulators, each with their own data path, so that they can be used simultaneously. Single 
Instruction-Multiple Data (SIMD) processors can then apply the same operation to several data operands 
at the same time (two simultaneous MACs for instance). An other way of implementing this, is to have 
the processor accommodate multiple date lengths, so that for instance a 16-bit multiplier can accomplish 
2 simultaneous 8-bit multiplications. A good example of this idea is the TigerSHARC processor from 
Analog Devices, which is illustrated in Figure 12.6. Figure 12.6(a) illustrates the general architecture of 
the processor core, with multiple buses and memory banks, multiple address generation units, and two 
parallel 32-bit computational units. Figure 12.6(b) shows how theses two computational units can be used 
in parallel, together with sub-word parallelism to enhance the throughput on sub-word operations: in the 
case illustrated, 8 16-bit MAC operations can be earned out in just one cycle. 

SIMD is efficient only for algorithms that lend themselves to this kind of parallel execution, and for instance 
not where data are treated in series or where a low-delay feedback loop exists. But it is very efficient for 
vector-intensive operations like multimedia processing. 


Instruction level parallelism: Instruction level parallelism is achieved by having different units with dif¬ 
ferent roles running concurrently. Two main families of parallel DSPs exist, namely those with VLIW (Very 
Long Instruction Word) architecture and those with super-scalar architectures. Super-scalar architectures 
are used heavily in general purpose processors (e.g. Intel’s Pentium), and contain a special scheduling unit 
which determines at run-time which instructions could run in parallel based e.g. on data dependencies. As 
is, this kind of behaviour is not desirable in DSPs, because the run-time determination of parallelism makes 
the predictability of execution time difficult — though it is a factor of paramount importance in real time 
operation. This is why there are few super-scalar DSPs available. 

VLIW architectures [6] are characterized by long instruction words, which are in fact concatenations of 
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(a) (b) 


Fig. 12.6 Simplified view of an Analog Devices TigerSHARC architecture, (a) The figure 
shows the 3 128-bit buses and 3 memory banks as well as 2 computational units and 2 address 
generation units. The programmer can assign program and/or data to each bus/memory bank, 
(b) Illustration of the data-level parallelism by SIMD operation: each computational unit can 
execute two 32-bit x 32-bit fixed-point MACs per cycle, but can also work in sub-word paral¬ 
lelism, executing up to four 16-bit x 16-bit MACs, for a total of eight simultaneous 16-bit x 
16-bit MACs per cycle. ( Source: ADI [1 ]) 


several shorter instructions. Each short instruction will ideally execute in parallel with the others on a differ¬ 
ent hardware unit. This principle is illustrated in Figure 12.7. This heavy parallelism normally provides an 
enormous increase in instruction throughput,but in practise, some limitations will prevent the hardware units 
from working at full rate in parallel all the time. For instance, data dependencies between the short instruc¬ 
tions might prevent their parallel execution; also, if some of the short instructions in a VFIW instruction use 
the same resources (e.g. registers), they will not be able to execute in parallel. 

A good illustration of these limitations can be given by looking at a practical example of VFIW processor: 
the TMS320C6xxx family from Texas Instruments. The simplified architecture of this processor is shown 
in Figure 12.8, where the address buses have been omitted for clarity. In the TMS320C6xxx, the instruction 
words are 128-bit wide, each containing eight 32-bit instructions. Each of these 32-bit instructions is to be 
executed by one of the 8 different computational units. These 8 units are constituted by 4 duplicate units, 
grouped in so-called data paths A and B (see Figure 12.8): AFU, AFU and shifter, address generation unit 
and multiplier. Each data path contains a set of registers, and both data paths share the same data memory. 
It is clear in this case that in order for all the 8 instructions of a VFIW instruction to be executed in parallel, 
one instruction must be executed on each computational unit, i.e. for instance, the VFIW instruction must 
contain two multiplications. The actual 32-bit instructions from the same 128-bit VFIW that are executed in 
parallel are in fact chosen at assembly time, by a special directive in the TMS320C6xxx assembly language. 
Of course, if the program is written in some high-level language such as C, the compiler takes care of this 
instructions organization and grouping. 

Another example of processor using VFIW is the TigerSHARC from Analog Devices, described earlier (see 
Figure 12.6). This DSP combines SIMD and VFIW to attain a very high level of parallelism. VFIW means 
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Fig. 12.7 Principle of VLIW architecture: long instructions are split in short instructions 
executed in parallel by different hardware units. 


that each of the two computational units in Figure 12.6(b) can be issued a different instruction. 


Other Implementations 

We have reviewed the characteristics of DSPs, and shown why they are suited to the implementation of DSP 
algorithms. Flowever, it is obvious that some other choices are possible when a DSP algorithm has to be 
implemented in an end-product. These choices are listed below, and compared with a PDSP-based solution. 


General Purpose Processor: Nowadays, general purpose processors, such as Intel’s Pentium, show ex¬ 
tremely high clock rates. They also have very efficient caching, pipelining and make extensive use of 
super-scalar architectures. Most of them also have a SIMD extension to their instruction set (e.g. MMX for 
Intel). However, they suffer some drawbacks for DSP implementations: high power consumption and need 
for proper cooling; higher cost (at least than most fixed-point DSPs); and lack of execution-time predictabil¬ 
ity because of multiple caching and dynamic super-scalar architectures, which make them less suited to hard 
real-time requirements. 


ASIC: An Application Specific Integrated Circuit will execute the required algorithm much faster than a 
programmable DSP processor, but will of course lack any reconfigurability. So upgrade and maintenance of 
products in the field are impossible. The design costs are also prohibitive except for very high volumes of 
units deployed. 


FPGA: Field-Programmable Gate Arrays are hardware reconfigurable circuits that represent some middle 
way between a programmable processor and and ASIC. They are faster than PDSPs, but in general have 
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Fig. 12.8 Simplified architecture of VLIW processor: the Texas Instruments TMS320C6xxx 
VLIW Processor. Each VLIW instruction is made of eight 32-bit instructions, for up to eight 
parallel instructions issued to the eight processing units divided in operational Units A and B. 
( Source.TI [13]) 
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higher power consumption. They are more expensive than PDSPs. The design work tends to be also longer 
than on a PDSP 


12.4 Benchmarking PDSPs 

Once the decision has been made to implement an algorithm on a PDSP, the time comes to choose a model 
amongst the many available brands and families. Many factors will enter into account, and we try to review 
some of these here. 


Fixed or Floating Point: As stated earlier, floating point DSPs have higher dynamic range and do not 
require the programmer to worry as much about overflow and round-off errors. These comes at the expense 
of a higher cost and a higher power consumption. So, most of the time, the application will decide: applica¬ 
tions where low-cost and/or low power consumption arc an issue (large volume, mobile,...) will call for a 
fixed-point DSP. Once the type of arithmetic is chosen, one still has to choose the word width (in bits). This 
is crucial for fixed-point application, since the number of bits available directly defines the dynamic range 
attainable. 


Speed: Constructors publish the speed of their devices as MIPS, MOPS or MFLOPS. MIPS (Millions 
of instructions per second) gives simply a hypothetical peak instruction rate through the DSP. It is simply 
determined by 

Ni_ 

T, ’ 

where 7) is the processor’s instruction cycle time in ps, and Nj is the maximum number of instructions exe¬ 
cuted per cycle (Nj > 1 for processors with parallelism). Due to the big differences between the instruction 
sets, architectures and operational units of different DSPs, it is difficult to conclude from MIPS measure¬ 
ments. Instructions on one DSP can be much more powerful than on another one, so that a lower instruction 
count will do the same job. 

MOPS or MFLOPS (Millions of Operations per Second or Millions of Floating-Point Operations per Sec¬ 
ond) have no consensual definition: no two device manufacturers do agree on what an “operation” might 
be. 

The only real way of comparing different DSPs in terms of speed is to run highly optimized benchmark 
programs on the devices to be compared, including typical DSP operations: FIR filtering, FFT,... Figure 12.9 
shows an example of such a benchmark, the BDTIMark2000, produced by Berkeley Design Technologies. 


Design issues: Time-to-market is an important factor that will perhaps be the main driving force towards 
the choice of any DSP. In order to ease the work of design engineers, the DSP manufacturer must provide 
development tools (compiler, assembler). Since most new DSPs have then - own instruction set and archi¬ 
tecture, if prior knowledge of one DSP is available, it may orient the choice of a new DSP to one that is 
code-compatible. Also, in order to reuse legacy code, designers may opt for a DSP that is code-compatible 
with one they already use. 
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Fig. 12.9 Speed benchmark for several available DSPs. The benchmark is the BDTI- 
Mark2000, by Berkeley Design Technologies (higher is faster). BDTIMark2000 is an aggregate 
speed benchmark computed from the speed of the processor in accomplishing common DSP 
tasks such as FFT, FIR filtering, HR filtering,... ( source: BDTI) 


New high-level development tools should ease the development of DSP algorithms and their implementa¬ 
tion. Some manufacturers have a collaboration e.g. with Mathworks to develop a MATLAB/SIMULINK 
toolbox enabling direct design and/or simulation of software for the DSP chip in MATLAB and SIMULINK. 

12.5 Current Trends 

Multiple DSPs: Numerous new applications cannot be tackled by one DSP processor, even the most 
heavily parallel ones. As a consequence, new DSP chips are developed that are easily grouped in parallel 
or in clusters to yield heavily parallel computing engines [2]. Of course, new challenges appear in the 
parallelisation of algorithms, shared resources management, and resolution of data dependencies. 


FPGAs coming on fast: FPGA manufacturers are aggressively entering the DSP market, due mainly to 
the very high densities attained by their latest products and to the availability of high-level development 
tools (directly from a block-level vision of the algorithm to the FPGA). The claimed advantages of FPGAs 
over PDSPs are speed and a completely flexible structure that allows for complete customization and highly 
parallel implementations [12]. For example, one manufacturer now promises the capability of implementing 
192 eighteen-bit multiplications in parallel. 
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Chapter 13 

Multirate systems 

13.1 Introduction 

Multi-rate system: A multi-rate DSP system is characterized by the use of multiple sampling rates in its 
realization, that is: signal samples at various points in the systems may not correspond to the same physical 
sampling frequency. 

Some applications of multi-rate processing include: 

• Sampling rate conversion between different digital audio standards. 

• Digital anti-aliasing filtering (used in commercial CD players). 

• Subband coding of speech and video signals 

• Subband adaptive filtering. 

In some applications, the need for multi-rate processing comes naturally from the problem definition (e.g. 
sampling rate conversion). In other applications, multi-rate processing is used to achieve improved perfor¬ 
mance of a DSP function (e.g. lower binary rate in speech compression). 


Sampling rate modification: The problem of sampling rate modification is central to multi-rate signal 
processing. Suppose we are given the samples of a continuous-time signal x a (t), that is 

x[n] = x a (nT s ), n £ Z (13.1) 

with sampling period T s and corresponding sampling frequency F s = 1 /T s . Then, how can we efficiently 
generate a new set of samples 

x[n\=x a {nT'), n€Z (13.2) 

corresponding to a different sampling period, i.e. T' / T S 1 

Assuming that the original sampling rate F s > 2 F max , where F max represent the maximum frequency con¬ 
tained in analog signal x a (t), one possible approach is to reconstruct x a (t) from the set of samples x[n\ (see 
Sampling Theorem, Section 7.1.2) and then to resample x a (t) at the desired rate F' = 1 fT'. This approach 
is expensive as it requires the use of D/A and A/D devices. 
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In this Chapter, we seek a cheaper, purely discrete-time solution to the above problem, i.e. performing the 
resampling by direct manipulation of the discrete-time signal samples x[n\, without going back in the analog 
domain. 

The following special terminology will be used here: 

• T' > T s (i.e. F' < F s ) =x downsampling 

• T’ < T s (i.e. F' > F s ) =>• upsampling or interpolation 

13.2 Downsampling by an integer factor 

Let T' = MT S where M is an integer > 1. In this case, we have: 

x'[n] = x a {nT') 

= x a (nMT s ) = x[nM\ (13.3) 

The above operation is so important in multi-rate signal processing that it is given the special name of 
decimation. Accordingly, a device that implements 13.3 is referred to as a decimator. The basic properties 
of the decimator are studied in greater details below. 


M-fold decimation: 

Decimation by an integer factor M > 1, or simply M-fold decimation, is defined by the following operation: 

xd [n] = d)M{x[n]} =x[nM\, «GZ (13.4) 

That is, the output signal xo[n\ is obtained by retaining only one out of every M input samples x\n). 

The block diagram form of the M-fold decimator is illustrated in Figure 13.1. The decimator is sometimes 


x\n\ 
o— 


rate = 1 IT 


M i 


x D [n] 
—o 


rate = \/MT 


Fig. 13.1 M-fold decimator 

called a sampling rate compressor or subsampler. It should be clear from (13.4) that the decimator is a linear 
system, albeit a time-varying one, as shown by the following example. 


Example 13.1: 

► Consider the signal 
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If x[n] is applied at the input of a 2-fold decimator (i.e. M = 2), the resulting signal will be 

xd[u\ = x[2n] 

= cos(7tn) u[2n\ 

= {...,0,0,1,-i, 

Only those samples of x[n] corresponding to even values of n are retained in x/j [n]. This is illustrated in 
Figure 13.2. 


x[n] 


i 

2 


0 1 

3 4 

n 




x[«] = x{n — 1] 


i 


3 


’ 

1 2 

4 

n 


x D [n 

*x D [n- 1] 


i 


0 1 2 3 4 n 


Fig. 13.2 Example of 2-fold decimation 


Now consider the following shifted version of x\n]: 

x[n] = x[n — 1] 

= {..., 0 , 0 , 1 , 0 ,- 1 , 0 , 1 , 0 ,- 1 ,...} 

If x\n\ is applied at the input of a 2-fold decimator, the output signal will be 

xo[n\ = x\2n\ 

= {•■•, 0 , 0 , 0 , 0 , 0 , 0 ,...} 

Note that xo[n\ ^ Xo[n — 1]. This shows that the 2-fold decimator is time-varying. ◄ 


Property: Let xo[n\ = x\nM\ where M is a positive integer. The ^-transforms of the sequences xo[n] and 
x[n], denoted Xd(z) and X(z), respectively, are related as follows: 


i M -1 

M k =0 


(13.5) 


where Wm = e J 2k / m . 
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Proof: Introduce the sequence 


p\l\ a! M y e j2nkl/M = ( 1, l = 0,±M,±2M,. 
m L 1 0, otherwise 


M 


k =o 


Making use of p[n], we have: 


Xd(z) = £ x D [n\z 

n——oo 


L MM* 


—l/M 

oo 1 M—i 

E © E 

——oo /:=0 

i M— 1 oo 

■;E E 


£=0 l — —oo 

M— 1 




4=0 


Remarks: 

• In the special case M = 2, we have W? = e~ 77t = — 1, so that 

X D (z)= l -(X(z 1 ' 2 )+X(-z 1/2 )) 


(13.6) 


(13.7) 


(13.8) 


Setting z = g /co in (13.5), an equivalent expression is obtained that relates the DTFTs of x ( /[n) and x[n], 
namely: 

, , 1 V f(o-2nk\ 

M<») = mL x I -ft- - I ( 13 -9) 


4=0 


M 


According to (13.9), the spectrum Xo(co) is made up of properly shifted, expanded images of the 
original spectrum X(co). 


Example 13.2: 

► In this example, we illustrate the effects of an M-fold decimation in the frequency domain. Consider a 
discrete-time signal x[n\ with DTFT X (co) as shown in Figure 13.3(top), where (O max = Jt/2 denotes the 
maximum frequency contained in x[n}. That is, A - (or) = 0 if CO max < |co| < 71. Let x/fn] denote the result 
of a M-fold decimation on x[n}. 

First consider the case M = 2, where (13.9) simplifies to 

A 0 (co) = i[A(|)+A(^^)] (13.10) 
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X(a>) 





X D (co) M - 2 



Fig. 13.3 


In this equation, X(co/2) is a dilated version of X (co) by a factor of 2 along the co-axis. Note that X (co) is 
no longer of period 2n but instead its period is 47t. Besides scaling factor 1 /2, A'/)(ojj is obtained as the 
sum of X(co/2) and its shifted version by 2jt, i.e. X ( fl) © ). The introduction of this latter component 
restores the periodicity to 2 ji. The resulting DTFT Xd((o) is illustrated in Figure 13.3 (middle). 

Note that in the above case M = 2, the two sets of spectral images X((o/2) and X ( m ^ 2jt ) do not overlap 
because (D max < pi/2. Thus, there is no spectral aliasing between the images and, besides the amplitude 
scaling and the frequency dilation, the original spectral shape 2f(co) is not distorted. In other words, no 
information about the original x[n] is lost through the decimation process. We will see later how x[n] can 
be recovered from x/j [n] in this case. 

Next consider the case M — 3, where (13.9) simplifies to 

x D (a>) = *[x(j)+x( (0 ^ 2k )+x( ( 0 ^ 4k )] (13.11) 

Here, Xd(co) is obtained by superposing dilated and shifted versions of X (co), as shown in Figure 13.3 
(bottom). Here the dilation factor is 3 and since (0 max > Jt/3, the various images do overlap. That is, 
there is spectral aliasing and a loss of information in the decimation process. ◄ 


Discussion: It can be inferred from the above example that if X(oo) is properly band-limited, i.e. 

X(co) = 0 for ^<M< n (13.12) 

then no loss of information occurs during the decimation process. In this special case, the relationship (13.9) 
between X(co) andX/)(co) simplifies to the following if one only considers the fundamental period [— 

X D ((D) = ±X(^), 0<|co|<7t (13.13) 
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Downsampling system: In practice, a low-pass digital filter with cut-off at %/M would be included along 
the processing chain prior to M-fold decimation. The resulting system, referred to as a downsampling system 
is illustrated in Figure 13.4. 



Fig. 13.4 


Ideally, H E ( oo) is a the low-pass anti-aliasing filter H E { to) should have the following desired specifications: 

1 if 1001 < %/M 


H d { oo) = 


0 if %/M < |GO| < %. 


(13.14) 


13.3 Upsampling by an integer factor 

Here we consider the sampling rate conversion problem (13.1)- (13.2) with T' = T S /L where L is an integer 

> 1 . 

Suppose that the underlying analog signal x a (t) is band-limited to fly, i.e. |X(fl)| = 0 for fl > kl N , and 
that the sequence x[n] was obtained by uniform sampling of x a (t) at a rate F s such that fl v = 2%F S > 2 fly. In 
this case, according to the sampling theorem (Section 7.1.2), x a {t) can be expressed in terms of its samples 
x[n] as 

oo j 

x a (t)= Y *Msinc(— -k) (13.15) 

k——oo s 

Invoking (13.2), the new samples x'[n] can therefore be expressed in terms of the original samples x\n] as 
follows: We have 


x[n\ = x a (nT/) 

“ /7 T' 

= Y x[k}sinc(—f-k) 

k——oo s 

00 n — Id 

= Y *[&]sinc(—-—) (13.16) 

k=-°° L 

The above equation can be given a simple interpretation if we introduce the concept of sampling rate expan¬ 
sion. 


13.3.1 L-fold expansion 


Sampling rate expansion by an integer factor L > 1, or simply L-fold expansion, is defined as follows: 


x E [n\ = U L {x[n}} 



(13.17) 
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That is, L — 1 zeros are inserted between each consecutive samples of x[n\. Real-time implementation of an 
L-fold expansion requires that the output sample rate be L times larger than the input sample rate. 

The block diagram form of an L-fold expander system is illustrated in Figure 13.5. The expander is also 
sometimes called upsampler in the DSP literature. 


x[n] 
o -1 

rate = 1 / L 


L T 


x E [n ] 

♦-o 

rate = L/T s 


Fig. 13.5 L-fold expander. 

Sampling rate expansion is a linear time-varying operation. This is illustrated in the following example. 


Example 13.3: 

► Consider the signal 

x[n\ = a n u[n\ 

= {... ,0,0,1,a,a 2 ,a 3 ,...} 

L-fold expansion applied to x[n\, with L = 3, yields 

x E [n ] = {• •. ,0,0,0,1,0,0,a,0,0, a 2 , 0,0,a 3 ,0,0,...} 

That is, two zero samples are inserted between every consecutive samples of x[n}. 

Next, consider the following shifted version of x[ri\: 

x[n] = x[n — 1] 

= {... ,0,0,0, l,a,fl 2 ,a 3 ,...} 

The result of a 3-fold expansion on x[n] is 

x E [n) = {... ,0,0,0,0,0,0,1,0,0,a,0,0,a 2 ,0,0,a 3 ,0,0,...} 

Note that x E [n] fx E [n — 1], showing that the expander is a time-varying system. Actually, in the above 
example, we have x E [n] = x E [n — 3]. ◄ 


Property: Let x E [n] be defined as in (13.17), with L a positive integer. The z-transforms of the sequences 
x E [n] and x[n], respectively denoted X E {z) and A(z), are related as follows: 

X e (z)=X(z l ) (13.18) 

Proof: Since the sequence x E [n] = 0 unless n = kL, k e Z, we immediately obtain 

oo oo 

x e{z) = Yj x E [n]z~ n = Y x E [kL\z~ kL 

n——oo k—— oo 

oo 

= Y x[k]z~ kL = X{z L ) □ (13.19) 

k =—oo 
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Remarks: Setting z = <? 7 “ in (13.18), we obtain an equivalent relation in terms of the DTFTs of x E \n] and 
x[n], namely: 

X E ((0) =X(coL) (13.20) 

According to (13.20), the frequency axis in X(co) is compressed by a factor L to yield X E ( co). Because of 
the periodicity of X (co) this operation introduces L — 1 images of the (compressed) original spectrum within 
the range [— Jt,7t]. 

Example 13.4: 

► The effects of an L-fold expansion in the frequency domain are illustrated in Figure 13.6. The top portion 


X(co) . 




Fig. 13.6 Effects of L-fold expansion in the frequency domain. 

shows the DTFT X(co) of the original signal x[n\. The bottom portion shows the DTFT Xe(co) of x E [n\, 
the L-fold expansion of x[n\, in the special case L = 4. As a result of the time domain expansion, the 
original spectrum X(co) has been compressed by a factor of L = 4 along the co-axis. ◄ 


13.3.2 Upsampling (interpolation) system 


We are now in a position to provide a simple interpretation for the upsampling formula (13.16). 

Suppose x a {t) is BL with maximum frequency fly. Consider uniform sampling at rate F> v > 2F>,y followed 
by L-fold expansion, versus direct uniform sampling at rate Q! s = LQ. S . The effects of these operations in the 
frequency domain are shown in Figure 13.7 in the special case L = 2 and Fl s = 2fly. 

Clearly, X'fco) can be recovered from X E (a >) via low-pass filtering: 

X'(co) = tf/( oo)X £ (co) (13.21) 


where ///(to) is an ideal low-pass filter with specifications 



if |co| < 7t/L 
if n/L < |co| < 7t 


(13.22) 
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L T 



Fig. 13.7 Sampling at rate Cl s = 2£Ty followed by L-fold expansion (left) versus direct sam¬ 
pling at rate Q! s = (right). 
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The equivalent impulse response of this filter is given by 

hi[n] = sinc(^) (13.23) 

Let us verify that the above interpretation of upsampling as the cascade of a sampling rate expander followed 
by a LP filter is consistent with (13.16): 

“ n — kl 

x[n\ = Y x[k]sinc(—-—) 

k=~oo L 

“ n — kl 

= Yj XE sinc ( T -) 

k=— oo 

OO 7 

= Y x e [ 1 ] sinc(^-) 

/—— OO 

oo 

= Y XE (13-24) 
1——00 

A block diagram of a complete upsampling system operating in the discrete-time domain is shown in Fig¬ 
ure 13.8. We shall also refer to this system as a digital interpolator. In the context of upsampling, the LP 



Fig. 13.8 Discrete-Time Upsampling System, 
filter ///(go) is called anti-imaging filter, or also interpolation filter. 


13.4 Changing sampling rate by a rational factor 


Suppose and <T V are related as follows: 

^ (13-25) 

Such a conversion of the sampling rate may be accomplished by means of upsampling and downsampling 
by appropriate integer factors: 


ft,. 


Upsampling 

->■ Lft, 

by L 


Downsampling 
-> 

by M 



(13.26) 


The corresponding block diagram is shown in Figure 13.9. To avoid unnecessary loss of information when 
M > 1, upsampling is applied first. 


Simplified structure: Recall the definitions of ///(go) and ///>( go): 



I GO I < %/L 
%/L < | GO| < 71 


(13.27) 
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upsampling by L downsampling by M 


Fig. 13.9 Sampling rate alteration by a rational factor. 



|go| < %/M 
%/M < |co| < % 


The cascade of these two filters is equivalent to a single LP filter with frequency response: 


H id ((o) = H I (a)H D ( co) 


L, |co| < to c 

0, C0 c < |co| < 71 


where 


C0 c = min(7t/M,7t/L) 


The resulting simplified block diagram is shown in Figure 13.10. 


(13.28) 


(13.29) 

(13.30) 



Fig. 13.10 Simplified rational factor sampling rate alteration 


13.5 Polyphase decomposition 

Introduction: The polyphase decomposition is a mathematical tool used in the analysis and design of 
multirate systems. It leads to theoretical simplifications as well as computational savings in many important 
applications. 

Consider a system function 

oo 

H(z)= £ h[n\z~ n (13.31) 

n ——oo 

where h[n\ denotes the corresponding impulse response. Consider the following development of H (V) where 
the summation over n is split into a summations over even odd indices, respectively: 

oo oo 

H(z) = £ h[2r\ Z - 2r + £ h[2r+l] Z - {2r+l) 

Y ——oo Y —— 00 

oo oo 

= £ h[2r}z~ 2r + z- 1 £ h[2r+ l\z~ 2r 

Y— —oo Y —— 00 

= E Q (z 2 ) + z- l E x (z 2 ) (13.32) 
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where the functions Eq(z) and E\ (z) arc defined as 


OO 


Eo(z) = 

£ h i 2r ]z r 

r——oo 

(13.33) 

Ei(z) = 

oo 

£ h[2r + l]z~ r 

(13.34) 


r ——oo 


We refer to (13.32) as the 2-fold polyphase decomposition of the system function H(z). The functions Eq(z) 
and E\(z) are the corresponding polyphase component. The above development can be easily generalized, 
leading to the following result. 

Property: In its region of convergence, any system function H(z) admits a unique M- fold polyphase 
decomposition 

M—1 

H(z) = £ z~ l Ei(z M ) (13.35) 

/=o 

where the functions Efz) are called polyphase components of H(z). 

Proof: The basic idea is to split the summation over n in (13.31) into M individual summation over index 
r E Z, such that each summation covers the time indices n = Mr + /, where / = 0, — 1: 

M—1 OO 

H ( z ) = £ £ h[Mr + l] Z - {Mr+l) 

1=0 r =-°° 

M -1 

= £ z ~' H h[Mr + l\z~®**} 

1=0 r =—°° 

M -1 

= £ z^^/(z M ) (13.36) 

1=0 


where we define 

oo 

Ei(z)= £ A[Mr + /]z" r (13.37) 

r——oo 

This shows the existence of the polyphase decomposition. The unicity is proved by involving the unique 
nature of the series expansion (13.31). 


Remarks: 

• The proof is constructive in that in suggests a way of deriving the polyphase components Efz :) 
via (13.37). 

• The use of (13.37)is recommended for deriving the polyphase components in the case of an FIR filter 
H(z). For HR filters, there may be simpler approaches. 

• In anycase, once a polyphase decomposition as in (13.36) has been obtained, one can be sure that it is 
the right one since it is unique. 
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Example 13.5: 

► Consider the FIR filter 

To find the polyphase components for M = 2, we may proceed as follows: 

H{z) = 1 — 6z -1 + 15z -2 — 20z -3 + 15z~ 4 — 6z~ 5 +z~ 6 

= 1 + 15z“ 2 + 15z“ 4 + z~ 6 + z“ 1 (-6 - 20z -2 - 6z~ 4 ) 

= £'o(z 2 )+z“ 1 £i(z 2 ) 

where 

Eq{z) = 1 + 15z _1 + 15z~ 2 + z“ 3 
£ 1 (z) = -6-20z _1 -6z“ 2 


Noble identities: The first identy can be used to exchange the order of a decimator and an arbitrary filter 
G(z): The second identity can be used to exchange the order of an expander and filter G(z): The proof of 



Fig. 13.11 1st Noble identity. 



Fig. 13.12 2nd Noble identity, 
these identities are beyond the scope of this course. 


Efficient downsampling structure: The polyphase decomposition can be used along with Noble identity 
1 to derive a computationally efficient structure for downsampling. 

• Consider basic block diagram of a downsampling system: Suppose that the low-pass anti-aliasing 



Fig. 13.13 Block diagram of a downsampling system, 
filter H(z) is FIR with length N, i.e. 

H(z) = £ h[n]z~ n (13.38) 

n =0 
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Assuming a direct form (DF) realization for H(z), the above structure requires N multiplications per 
unit time at the input sampling rate. Note that only 1 out of every M samples computed by the FIR 
filter is actually output by the M -fold decimator. 

• For simplicity, assume that A is a multiple of M. Making use of the polyphase decomposition (13.36), 
the following equivalent structure for the downsampling system is obtained where the polyphase com¬ 



es 


o o p 



O 


'’z 1 


n 


Z7 f-M \ 



Em- iv z ) 



Fig. 13.14 Equivalent structure for the downsampling system. 


ponents are given here by 


E,(z M ) 


N/M-l 

£ h[Mr + l\z~ Mr 


r =0 


(13.39) 


If realized in direct form, each filter E/(z M ) requires N/M multiplications per unit time. Since there 
are M such filters, the computational complexity is the same as above. 

• Finally, consider interchanging the order of the filters Ei(z m ) and the decimator in Figure 13.14 by 
making use of Noble identity 1: Observe that the resulting structure now only requires N/M multipli¬ 



es 



Fig. 13.15 Final equivalent structure for the downsampling system, 
cations per unit time. 
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One could argue that in the case of an FIR filter H(z), the computational saving is artificial since actually, 
only one out of every M filter outputs in Fig. 13.13 needs to be computed. Flowever, since the input samples 
are coming in at the higher rate F s , the use of a traditional DF realization for H(z) still requires that the 
computation of any particular output be completed before the next input sample x[n] arrives and modifies 
the delay line. Thus the peak required computational rate is still N multiplies per input sample, even though 
the filtering operation needs only be activated once every M samples. This may be viewed as an inefficient 
use of available computational resources. The real advantage of the structure in Fig. 13.15 is that it makes 
it possible to spread the FIR filter computation over time, so that the required computational rate is uniform 
and equal to N/M multiplications per input samples. 
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Chapter 14 

Applications of the DFT and FFT 


In this Chapter, we discuss two basic applications of the DFT and associated FFT algorithms. 

We recall from Chapter 6 that the linear convolution of two time-limited signals can be realized via circular 
convolution in the DFT domain. The first application describes an efficient way of realizing an FIR filtering 
via block DFT processing. Essentially, this amounts to decomposing the input signal into consecutive blocks 
of smaller length, and implementing the filtering in the frequency domain via FFT and IFFT processing. 
This results in significant computational savings, provided a processing delay is tolerated in the underlying 
application. 

In the second application, we look at the use of the DFT (efficiently computed via FFT) as a frequency 
analysis or estimation tool. We review the fundamental relationship between the DFT of a finite signal 
and the DTFT and we investigate the effects of the DFT length and windowing function on the spectral 
resolution. The effects of measurement noise on the quality of the estimation are also discussed briefly. 


14.1 Block FIR filtering 


Consider a causal FIR filtering problem as illustrated in Figure 14.1. Assume that the filter’s impulse re- 


x[n\ 

o— 


causal 
FIR filter 

y[n\ = h[n]*x[n] 
*■--o 


h[n\ 


Fig. 14.1 Causal FIR filtering 

sponse h[n] is time-limited to 0 <n<P — I and that the input signal x[n] is zero for n < 0. Accordingly, the 
filter output 

p-i 

y[n\ = h[n] *x[n\ = ^ /r[k].>c[/i — k] (14.1) 

k =0 

is also zero for n < 0. If x[n ] was also time-limited, then the DFT approach in Section 6.5.1 could be used 
to compute y[n], provided the DFT size N is large enough. 
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Two fundamental problems exist with respect to the assumption that x[n] is time-limited: x[n \: 

• in certain applications, it is not possible to set a specific limit on the length of x[n] (e.g. voice com¬ 
munications); 

• such a limit, if it exists, may be too large, leading to various technical problems (processing delay, 
memory requirement, etc.). 

In practice, it is still possible to compute the linear convolution (14.1) as a circular convolution via the DFT 
(i.e. FFT), but it is necessary to resort to block convolution. 


Block convolution: The basic principle of block convolution may be summarized as a series of three steps 
as follows: 

(1) Decompose x[n] into consecutive blocks of finite length L\ 

x[n]=^x r [n], r = 0,1,2,... (14.2) 

where x r [n] denotes the samples in the rthe block. 

(2) Filter each block individually using the DFT approach: 

x r [n ] =>y r [«] = li[n] *x r [n] (14.3) 

In practice, the required DFTs and inverse DFT are computed using an FFT algorithm (more on this 
later). 

(3) Reassemble the filtered blocks (as they become available) into an output signal: 

y r [n\^y[n] (14.4) 

The above generic steps may be realized in a different ways. In particular, there exist two basic methods of 
block convolution: overlap-add and overlap-save. Both will be described below. 


14.1.1 Overlap-add method: 

In this method, the generic block convolution steps take the following form: 


(1) Blocking of input data into non-overlapping blocks of length L: 

f x\n + rL] 0 < n < L , 

x r [n\ = < 

0 otherwise. 


(14.5) 


where, for convenience of notation, the shift factor rL is introduced so that the non-zero samples of 
x r [n] are all between 0 < n < L — 1. 

(2) For each input block x r [n], compute its linear convolution with FIR filter impulse response h\n] (see 
DFT approach below): 

y r [n\ = h[n] *v r [«] (14.6) 

Note that each block y r [n\ is now of length L + P — 1. 
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(3) Form the output signal by adding the shifted filtered blocks: 

oo 

y[n] = Y,yr[n-rL] (14.7) 

r =0 

The method takes its name from the fact that the output blocks y r [n] need to be properly overlapped and 
added in the computation of the desired output signal y[n\. 

Mathematical justification: It is easy to verify that the overlap-add approach produces the desired con¬ 
volution results in (14.1). Under the assumption that jc[«] = 0 for n < 0, we have from (14.5) that 

oo 

x\n\ = ^x r [n — rL] (14.8) 

r =0 

Then, invoking basic properties of the linear convolution, we have 

oo 

y[n] = h[n\ *x\n\ = h[n] * X x r [n — rL\ 

r=0 

oo oo 

= X h M * Xr [ ?I - rL] = X y r H 

r =0 r =0 

(14.9) 


DFT realization of step (2): Recall that the DFT approach to linear convolution amounts to computing 
a circular convolution. To ensure that the circular and linear convolutions are equivalent, the DFT length, 
say N, should be sufficiently large and the data must be appropriately zero-padded. Specifically, based on 
Section 6.5, we may proceed as follows 

• Set DFT length toA^>L + F — 1 (usually a power of 2) 

• Compute (once and store result) 

H[k\ =FFT A r{/?[0],...,/z[F-l],0,...,0} (14.10) 

• For r = 0,1,2,..., compute 

X r [k] = FFT#{x r [0],...,.* r [L— 1], 0,..., 0} (14.11) 

y r [n] = IFFTa ,{H[k]X r [k}} (14.12) 

Often, the value of N is selected as the smallest power of 2 that meets the above condition. 


Computational complexity: The computational complexity of the overlap-add method can be evaluated 
as follows. For each block of L input samples, we need 

• t log, N multiplications to compute X r [k] in (14.11) 

• N multiplications to compute H[k]X r \k\ in (14.12) 
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• log 2 A multiplications to compute y r [7?] in (14.12) 

for a total of N\og 2 N + N multiplications per L input samples. Assuming P ss L for simplicity, we may set 
N = 2L. The resulting complexity of the overlap-add method is then 

21og 2 L + 4 

multiplications per input sample. Linear filtering based on (14.1) requires P multiplications per input sample. 
Thus, a significant gain may be expected when P rs L is large (e.g. L = 1024 =>- 21og 2 L + 4 = 24 <L. 


Example 14.1: Overlap-add 

► The overlap-add process is depicted in Figures 14.2 and 14.3. The top part of Figure 14.2 shows the FIR 
filter h[n\ as well as the input signal x[n], supposed to be semi-infinite (right-sided). The three bottom 
parts of Figure 14.2 show the three first blocks of the input xo [n], xi [n] and x 2 [«]. Figure 14.3 shows the 
result of the convolution of these blocks with h[n\. Notice that the convolution results are longer than the 
original blocks. Finally, the bottom plot of Figure 14.3 shows the reconstructed output signal obtained 
by adding up the overlapping blocks y r [n\. ◄ 


h[ n ] 



0 5 10 15 20 25 30 35 40 

Fig. 14.2 Illustration of blocking in the overlap-add method. Top: the FIR filter h[n\\ Below: 
the input signal x[n\ and different blocks x r [n\. 
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yj n] 



0 5 10 15 20 25 30 35 40 

Fig. 14.3 Illustration of reconstruction in the overlap-add method. Top: several blocks y r [n\ 
after convolution; Bottom: the output signal y[«] . 


© B. Champagne & F. Labeau 


Compiled November 24, 2004 












































264 


Chapter 14. Applications of the DFT and FFT 


14.1.2 Overlap-save method (optional reading) 

This method contains the following steps: 

(1) Blocking of input data in overlapping blocks of length L, overlapping by P — 1 samples: 

\x[n + r{L — P + 1) — P + 1] 0 <n<L, 

Xp \ft\ — \ 

0 otherwise. 


(14.13) 


(2) For each input block, compute the L-point circular convolution with the FIR filter impulse response 
h[n\: 

y r [n] = h[n]®Xr[n\ (14.14) 

Note that each block y r [n] is still of length L. Since this circular convolution does not respect the 
criterion in (6.63), there will be time-aliasing. By referring to the example in Figure 6.12, one sees 
that the first P — 1 samples of this circular convolution will be corrupted by aliasing, whereas the 
remaining L — P + 1 will remain equal to the true samples of the corresponding linear convolution 
x r [n\ *h[n\. 

(3) Form the output signal by adding the shifted filtered blocks after discarding the leading P— 1 samples 
of each block: 

oo 

y[n] = £^y r [n ~ r(L — P+ 1 )\(u[n — P — r(L — P-\- 1)] — u[n — L — r(L — P + 1)]) (14.15) 

r =0 


Example 14.2: Overlap-save 

► The overlap-save process is depicted in Figures 14.4 and 14.5. The top part of Figure 14.4 shows the 
FIR filter h[n] as well as the input signal x[n]. The three bottom parts of Figure 14.4 show the three first 
blocks of the input xo [n ], xi [n] and X 2 [«]. Figure 14.5 shows the result of the circular convolution of these 
blocks with h[n\. Notice that the convolution white dots correspond to samples comipted by aliasing. 
Finally, the bottom plot of Figure 14.5 shows the reconstructed output signal obtained by adding up the 
blocks y r [n] after dropping the aliased samples. ◄ 


14.2 Frequency analysis via DFT/FFT 

14.2.1 Introduction 

The frequency content of a discrete-time signal x[n] is defined by its DTFT: 


oo 

X(ot>) = £ x[n]e~ jan , COG!—7t,7t], 

n——oo 


(14.16) 


As pointed out in Chapter 6, the practical computation of X (co) poses the several difficulties: 

• an infinite number of mathematical operations is needed (the summation over n is infinite and the 
variable co is continuous). 
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h[n] 



Fig. 14.4 Illustration of blocking in the overlap-save method. Top: the FIR filter h [n ]; Below: 
the input signal x[n\ and different blocks x r [n]. 
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Fig. 14.5 Illustration of reconstruction in the overlap-save method. Top: several blocks y r \n\ 
after convolution; Bottom: the output signal y[n] . 
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• all the signals samples x[n\ from — °o to °o must be available 
Recall the definition of the DFT 


N -1 

X[k] = £ x[n\e- Jatn , k E {0,1,... ,2V — 1} (14.17) 

where go*. = 2%k/N. In practice, the DFT is often used as an approximation to the DTFT for the puipose of 
frequency analysis. The main advantages of the DFT over the DTFT are 

• finite number of mathematical operation 

• only a finite set of signal samples x[n] is needed 

If x[n] = 0 for n < 0 and n > N, then the DFT and the DTFT are equivalent concept. Indeed, as we have 
shown in Chapter 6, there is in this case a 1-to-l relationship between the DFT and the DTFT, namely: 

X[k]=X((o k ) (14.18) 

N -1 

X(co) = £ X[k]p{ 00 -cot) (14.19) 

*=0 

where P( to) is an interpolation function defined in (6.18). 

In general, however, we are interested in analyzing the frequency content of a signal x[n] which is not a 
priori time-limited. In this case, the DFT only provides an approximation to the DTFT. In this Section: 

• we investigate in further detail the nature of this approximation; 

• based on this analysis, we present and discuss modifications to the basic DFT computation that lead 
to improved frequency analysis results. 


14.2.2 Windowing effects 

Rectangular window: Recall the definition of a rectangular window of length N : 


w R [n] = 


1 , 

0 , 


0 < n <N — 1 
otherwise 


The DTFT of w[n] will play a central role in our discussion: 


(14.20) 


Wk((0 ) - V - e-M-W Sin(COjV/2) 

} h sin(co/2) 


(14.21) 


The corresponding magnitude spectrum is illustrated in Figure 14.6 for the N 
• a main lobe with 


Ago = 


2n 

~N 


16. Note the presence of: 

(14.22) 


• sidelobes with peak a amplitude of about 0.22 (-13dB) 
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Fig. 14.6 Magnitude spectrum of N = 16 point rectangular window (linear scale) 

Relation between DFT and DTFT: Define the windowed signal 

y[n] = wr [n]x[n] (14.23) 

The A-point DFT of signal x[n] can be expressed as the DTFT of y[n ]: 

N -1 

X[k] = £ x[n]e- jakn 

n— 0 

oo 

= £ Mn]x[n]e~^ 

n=— oo 

oo 

= L y[n]e- ja * n = Y(<o k ) (14.24) 

n——oo 

where Y (to) denotes the DTFT of y[n\. The above shows that the DFT can be obtained by uniformly sampling 
Y (to) at the frequencies to = C0/t. 


Nature of Y (to): Since y[n] is obtained as the product of x[n] and wr[h], we have from the multiplication 
property of the DTFT (Ch. 3) that 

T(oo) = DTFT{w R [n]x[n]} 

= — r X(tQ)W R (to-ty)dty (14.25) 

271 J-n 

That is, Y (to) is equal to the circular convolution of the periodic functions Wr(oo) and X(to). When comput¬ 
ing the A^-point DFT of the signal x\n], the windowing of x\n\ that implicitly takes place in the time-domain 
is equivalent to the periodic convolution of the desired spectrum X(co) with the window spectrum Wr(cq). 
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Example 14.3: 

► Consider the signal 

x[n) = e^ n , ngZ 

where 0 < 0 < n for simplicity. The DTFT of x\n] is given by 

X (to) = 27t £ 5 a (co-0-2jt/t) 

k=-oc 

In this special case, equations (14.24)- (14.25) yield 

X[k\ = ^~ r X(4>)w *((Ok - fldty = W R (0) k - 0) 

Thus, the DFT values X[k] can be obtained by uniformly sampling the shifted window spectrum Wr (co — 

0) at the frequencies to = co*. ◄ 

Effects of windowing: Based on the above example and taking into consideration the special shape of the 
window magnitude spectrum in Figure 14.6, two main effects of the time-domain windowing on the original 
DTFT spectrum X(go) may be identified: 

• Due to non-zero width of the main lobe in Figure 14.6, local smearing (or spreading) of the spectral 
details in X(go) occur as a result of the convolution with Wr(go) in (14.25). This essentially corre¬ 
sponds to a loss of resolution in the spectral analysis. In particular, if say X (co) contains two narrow 
peaks at closely spaced frequencies 0) and 02, it may be that Y (to) in (14.25) only contains a single 
wider peak at frequency (0i + 02)/2. In other words, the two peaks will merge as a result of spectral 
spreading. The resolution of the spectral analysis with DFT is basically a function of the window 
width Aco = 2n/N. Thus resolution can be increased (i.e. Ago smaller) by increasing N. 

• Due to the presence of sidelobes in Figure 14.6, spectral leakage in X(co) will occur as a result of 
the convolution with Wr(go) in (14.25). For instance, even if X(co) = 0 in some band of frequency, 
Y (go) will usually not be zero in that band due to the trailing effect of the sidelobes in the convolution 
process. The amount of leakage depends on the peak sidelobe level. For a rectangular winodw, this is 
fixed to -13dB. 


Improvements: Based on the above discussion, it should be clear that the results of the frequency analysis 
can be improved by using a different type of window, instead of the rectangular window implicit in (14.24). 
Specifically: 

JV-l 

X[k] = £ w[n]x[n]e- jakn , k G {0,1,... ,1V — 1} (14.26) 

n —0 

where w[n] denotes an appropriately chosen window. In this respect, all the window functions introduced in 
Chapter 9 can be used. In selecting a window, the following approach might be taken: 

• Select window so that peak sidelobe level is below acceptable level. 

• Adjust window length N so that desired resolution Ago is obtained. For most windows of interest 
Ago = cte/N where cte depends on the specific window type. 

The trade-offs involved in the selection of a window for spectral analysis are very similar to those use in the 
window method of FIR filter design (Ch. 9). 


© B. Champagne & F. Labeau 


Compiled November 24, 2004 




270 





271 


References 


[1] Analog Devices Inc., Norwood, MA. TIgerSHARC DSP Microcomputer Preliminary Technical data, 
2002 . 

[2] Ashok Bindra. Novel architectures pack multiple dsp cores on-chip. Electronic Design, December 
2001 . 

[3] Yu Hen Hu, editor. Programmable Digital Signal Processors: Architecture, Programming and Appli¬ 
cations. Marcel Dekker, 2002. 

[4] Emmanuel C. Ifeachor and Barrie W. Jervis. Digital Signal Processing, a practical approach. Prentice- 
Hall, New Jersey, 2nd edition edition, 2002. 

[5] Phil Lapsley, Jeff Bier, Amit Shoham, and Edward A. Lee. DSP Processor Funcamentals: Architec¬ 
tures and Features. IEEE Press, 1997. 

[6] Ravi A. Managuli and Yongmin Kim. VLIW processor architectures and algorithm mappings for DSP 
applications. In Hu [3], chapter 2. 

[7] James H. McClellan, Ronald W. Schafer, and Mark A. Yoder. DSP First: a multimedia approach. 
Prentice-Hall, 1998. 

[8] Sanjit K. Mitra. Digital Signal Processing, a computer-based approach. McGraw-Hill, 2nd edition 
edition, 2001. 

[9] Motorola, Inc., Denver, CO. DSP56300 Family Manual, 2000. 

[10] Allan V. Oppehneim, Ronald W. Schfafer, and J. R. Buck. Discrete-Time Signal Processing. Prentice- 
Hall, New Jersey, 2nd edition edition, 1999. 

[11] John G. Proakis and Dimitris G. Manolakis. Digital Signal Processing, Principles, algorithms and 
applications. Prentice-Hall, 3rd edition edition, 1996. 

[12] Russell Tessier and Wayne Burleson. Reconfigurable computing and digital signal processing: Past, 
present and future. In Hu [3], chapter 4. 


[13] Texas Instruments, Houston, TX. TMS320C6204 Data Sheet, 2000. 





